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Preface 


This new edition reflects the development of the field of hypothesis 
testing since the original book was published 27 years ago, but the basic 
structure has been retained. In particular, optimality considerations con- 
tinue to provide the organizing principle. However, they are now tempered 
by a much stronger emphasis on the robustness properties of the resulting 
procedures. Other topics that receive greater attention than in the first 
edition are confidence intervals (which for technical reasons fit better here 
than in the companion volume on estimation, TPE*), simultaneous in- 
ference procedures (which have become an important part of statistical 
methodology), and admissibility. A major criticism that has been leveled 
against the theory presented here relates to the choice of the reference set 
with respect to which performance is to be evaluated. A new chapter on 
conditional inference at the end of the book discusses some of the issues 
raised by this concern. 

In order to accommodate the wealth of new results that have become 
available concerning the core material, it was necessary to impose some 
limitations. The most important omission is an adequate treatment of 
asymptotic optimality paralleling that given for estimation in TPE. Since 
the corresponding theory for testing is less satisfactory and would have 
required too much space, the earlier rather perfunctory treatment has been 
retained. Three sections of the first edition were devoted to sequential 
analysis. They are outdated and have been deleted, since it was not possible 
to do justice to the extensive and technically demanding expansion of this 
area. This is consistent with the decision not to include the theory of 
optimal experimental design. Together with sequential analysis and survey 
sampling, this topic should be treated in a separate book. Finally, although 
there is a section on Bayesian confidence intervals, Bayesian approaches to 
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hypothesis testing are not discussed, since they play a less well-defined role 
here than do the corresponding techniques in estimation. 

In addition to the major changes, many new comments and references 
have been included, numerous errors corrected, and some gaps filled. I am 
greatly indebted to Peter Bickel, John Pratt, and Fritz Scholz, who furnished 
me with lists of errors and improvements, and to Maryse Loranger and Carl 
Schaper who each read several chapters of the manuscript. For additional 
comments I should like to thank Jim Berger, Colin Blyth, Herbert Eisenberg, 
Jaap Fabius, Roger Farrell, Thomas Ferguson, Irving Glick, Jan Hemelrijk, 
Wassily Hoeffding, Kumar Jogdeo, the late Jack Kiefer, Olaf Krafft, Wil- 
liam Kruskal, John Marden, John Rayner, Richard Savage, Robert Wijs- 
man, and the many colleagues and students who made contributions of 
which I no longer have a record. 

Another indebtedness I should like to acknowledge is to a number of 
books whose publication considerably eased the task of updating. Above all, 
there is the encyclopedic three-volume treatise by Kendall and Stuart, of 
which I consulted particularly the second volume, fourth edition (1979) 
innumerable times. The books by Ferguson (1967), Cox and Hinkley (1974), 
and Berger (1980) also were a great help. In the first edition, I provided 
references to tables and charts that were needed for the application of the 
tests whose theory was developed in the book. This has become less 
important in view of the four-volume work by Johnson and Kotz: Distribu- 
tions in Statistics (1969-1972). Frequently I now simply refer to the ap- 
propriate chapter of this reference work. 

There are two more books to which I must refer: 

A complete set of solutions to the problems of the first edition was 
published as Testing Statistical Hypotheses: Worked Solutions. [Kallenberg 
et al. (1984)]. I am grateful to the group of Dutch authors for undertaking 
this labor and for furnishing me with a list of errors and corrections 
regarding both the statements of the problems and the hints to their 
solutions. 

The other book is my Theory of Point Estimation [Lehmann (1983)], 
which combines with the present volume to provide a unified treatinent of 
the classical theories of testing and estimation, both by confidence intervals 
and by point estimates. The two are independent of each other, but cross 
references indicate additional information on a given topic provided by the 
other book. Suggestions for ways in which the two books can be used to 
teach different courses are given in comments for instructors following this 
preface. 

I owe very special thanks to two people. My wife, Juliet Shaffer, critically 
read the new sections and gave advice on many other points. Wei Yin Loh 
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read an early version of the whole manuscript and checked many of the new 
problems. In addition, he joined me in the arduous task of reading the 
complete galley proofs. As a result, many errors and oversights were 
corrected. 

The research required for this second edition was supported in part by 
the National Science Foundation, and I am grateful for the Foundation’s 
continued support of my work. Finally, I should like to thank Linda 
Tiffany, who converted many illegible pages into beautifully typed ones. 
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Preface to the First Edition 


A mathematical theory of hypothesis testing in which tests are derived as 
solutions of clearly stated optimum problems was developed by Neyman 
and Pearson in the 1930s and since then has been considerably extended. 
The purpose of the present book is to give a systematic account of this 
theory and of the closely related theory of confidence sets, together with 
their principal applications. These include the standard one- and two-sam- 
ple problems concerning normal, binomial, and Poisson distributions; some 
aspects of the analysis of variance and of regression analysis (linear hy- 
pothesis); certain multivariate and sequential problems. There is also an 
introduction to nonparametric tests, although here the theoretical approach 
has not yet been fully developed. One large area of methodology, the class 
of methods based on large-sample considerations, in particular x? and 
likelihood-ratio tests, essentially has been omitted because the approach and 
the mathematical tools used are so different that an adequate treatment 
would require a separate volume. The theory of these tests is only briefly 
indicated at the end of Chapter 7. 

At present the theory of hypothesis testing is undergoing important 
changes in at least two directions. One of these stems from the realization 
that the standard formulation constitutes a serious oversimplification of the 
problem. The theory is therefore being reexamined from the point of view of 
Wald's statistical decision functions. Although these investigations throw 
new light on the classical theory, they essentially confirm its findings. I have 
retained the Neyman-Pearson formulation in the main part of this book, 
but have included a discussion of the concepts of general decision theory in 
Chapter 1 to provide a basis for giving a broader justification of some of the 
results. It also serves as a background for the development of the theories of 
hypothesis testing and confidence sets. 

Of much greater importance is the fact that many of the problems, which 
traditionally have been formulated in terms of hypothesis testing, are in 
reality multiple decision problems involving a choice between several deci- 
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sions when the hypothesis is rejected. The development of suitable proce- 
dures for such problems is at present one of the most important tasks of 
statistics and is finding much attention in the current literature. However, 
since most of the work so far has been tentative, I have preferred to present 
the traditional tests even in cases in which the majority of the applications 
appear to call for a more elaborate procedure, adding only a warning 
regarding the limitations of this approach. Actually, it seems likely that the 
tests will remain useful because of their simplicity even when a more 
complete theory of multiple decision methods is available. 

The natural mathematical framework for a systematic treatment of 
hypothesis testing is the theory of measure in abstract spaces. Since intro- 
ductory courses in real variables or measure theory frequently present only 
Lebesgue measure, a brief orientation with regard to the abstract theory is 
given in Sections 1 and 2 of Chapter 2. Actually, much of the book can be 
read without knowledge of measure theory if the symbol {p(x)d(x) is 
interpreted as meaning either [p(x)dx or X p(x), and if the measure-theo- 
retic aspects of certain proofs together with all occurrences of the letters a.e. 
(almost everywhere) are ignored. With respect to statistics, no specific 
requirements are made, all statistical concepts being developed from the 
beginning. On the other hand, since readers will usually have had previous 
experience with statistical methods, applications of each method are indi- 
cated in general terms, but concrete examples with data are not included. 
These are available in many of the standard textbooks. 

The problems at the end of each chapter, many of them with outlines of 
solutions, provide exercises, further examples, and introductions to some 
additional topics. There is also given at the end of each chapter an 
annotated list of references regarding sources, both of ideas and of specific 
results. The notes are not intended to summarize the principal results of 
each paper cited but merely to indicate its significance for the chapter in 
question. In presenting these references I have not aimed for completeness 
but rather have tried to give a usable guide to the literature. 

An outline of this book appeared in 1949 in the form of lecture notes 
taken by Colin Blyth during a summer course at the University of Cali- 
fornia. Since then, I have presented parts of the material in courses at 
Columbia, Princeton, and Stanford Universities and several times at the 
University of California. During these years I greatly benefited from com- 
ments of students, and I regret that I cannot here thank them individually. 
At different stages of the writing I received many helpful suggestions from 
W. Gautschi, A. Heyland, and L. J. Savage, and particularly from Mrs. C. 
Striebel, whose critical reading of the next to final version of the manuscript 
resulted in many improvements. Also, I should like to mention gratefully 
the benefit I derived from many long discussions with Charles Stein. 
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It is a pleasure to acknowledge the generous support of this work by the 
Office of Naval Research; without it the book would probably not have 
been written. Finally, I should like to thank Mrs. J. Rubalcava, who typed 
and retyped the various drafts of the manuscript with unfailing patience, 
accuracy, and speed. 


E. L. LEHMANN 


Berkeley, California 
June 1959 


Comments for Instructors 


The two companion volumes, Testing Statistical Hypotheses (TSH) 
and Theory of Point Estimation (TPE), between them provide an introduc- 
tion to classical statistics from a unified point of view. Different optimality 
criteria are considered, and methods for determining optimum procedures 
according to these criteria are developed. The application of the resulting, 
theory to a variety of specific problems as an introduction to statistical 
methodology constitutes a second major theme. 

On the other hand, the two books are essentially independent of each 
other, (As a result, there is some overlap in the preparatory chapters; also, 
each volume contains cross-references to related topics in the other.) They 
can therefore be taught in either order. However, TPE is somewhat more 
discursive and written at a slightly lower mathematical level, and for this 
reason may offer the better starting point. 

The material of the two volumes combined somewhat exceeds what can 
be comfortably covered in à year’s course meeting 3 hours a week, thus 
providing the instructor with some choice of topics to be emphasized. A 
one-semester course covering both estimation and testing can be obtained, 
for example, by deleting all large-sample considerations, all nonparametric 
material, the sections concerned with simultaneous estimation and testing, 
the minimax chapter of TSH, and some of the applications. Such a course 
might consist of the following sections: TPE: Chapter 2, Section 1 anda 
few examples from Sections 2,3; Chapter 3, Sections 1-3; Chapter 4, 
Sections 1-4. TSH: Chapter 3, Sections 1-3, 5,7 (without proof of Theorem 
6); Chapter 4, Sections 1-7; Chapter 5, Sections 1-4,6-8; Chapter 6, 
Sections 1-6, 11; Chapter 7, Sections 1-3, 5-8, 11, 12; together with material 
from the preparatory chapters (TSH Chapter 1,2; TPE Chapter 1) as it is 
needed. 
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CH AP T ETRA 


The General Decision 


Problem 


1. STATISTICAL INFERENCE AND STATISTICAL 
DECISIONS 


The raw material of a statistical investigation is a set of observations; these 
are the values taken on by random variables X whose distribution Py is at 
least partly unknown. Of the parameter 6, which labels the distribution, it is 
assumed known only that it lies in a certain set Q, the parameter space. 
Statistical inference is concerned with methods of using this observational 
material to obtain information concerning the distribution of X or the 
parameter @ with which it is labeled. To arrive at a more precise formula- 
tion of the problem we shall consider the purpose of the inference. 

The need for statistical analysis stems from the fact that the distribution 
of X, and hence some aspect of the situation underlying the mathematical 
model, is not known. The consequence of such a lack of knowledge is 
uncertainty as to the best mode of behavior. To formalize this, suppose that 
a choice has to be made between a number of alternative actions. The 
observations, by providing information about the distribution from which 
they came, also provide guidance as to the best decision. The problem is to 
determine a rule which, for each set of values of the Observations, specifies 
what decision should be taken. Mathematically such a rule is a function 6, 
which to each possible value x of the random variables assigns a decision 
d = 6(x), that is, a function whose domain is the set of values of X and 
whose range is the set of possible decisions. 

In order to see how 8 should be chosen, one must compare the conse- 
quences of using different rules. To this end suppose that the consequence 
of taking decision d when the distribution of X is P, is a loss, which can be 
expressed as a nonnegative real number L(0, d). Then the long-term 
average loss that would result from the use of à in a number of repetitions 
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of the experiment is the expectation E[L(0, 8(X))] evaluated under the 
assumption that P, is the true distribution of X. This expectation, which 
depends on the decision rule 8 and the distribution Ps, is called the risk 
function of 8 and will be denoted by R(0, 5). By basing the decision on the 
observations, the original problem of choosing a decision d with loss 
function L(0, d) is thus replaced by that of choosing 8, where the loss is 
now R(6, 8). 

The above discussion suggests that the aim of statistics is the selection of 
a decision function which minimizes the resulting risk. As will be seen later, 
this statement of aims is not sufficiently precise to be meaningful; its proper 
interpretation is in fact one of the basic problems of the theory. 


2. SPECIFICATION OF A DECISION PROBLEM 


The methods required for the solution of a specific statistical problem 
depend quite strongly on the three elements that define it: the class 
P = (P,,0 € Q} to which the distribution of X is assumed to belong; the 
structure of the space D of possible decisions d; and the form of the loss 
function L. In order to obtain concrete results it is therefore necessary to 
make specific assumptions about these elements. On the other hand, if the 
theory is to be more than a collection of isolated results, the assumptions 
must be broad enough either to be of wide applicability or to define classes 
of problems for which a unified treatment is possible. 

Consider first the specification of the class . Precise numerical assump- 
tions concerning probabilities or probability distributions are usually not 
warranted. However, it is frequently possible to assume that certain events 
have equal probabilities and that certain others are statistically independent. 
Another type of assumption concerns the relative order of certain infinitesi- 
mal probabilities, for example the probability of occurrences in an interval 
of time or space as the length of the interval tends to zero. The following 
classes of distributions are derived on the basis of only such assumptions, 
and are therefore applicable in a great variety of situations. 

The binomial distribution b( p, n) with 


Q) r(x-x)-(i)ma-p'"",  x-9...,n, 0sp«1. 


This is the distribution of the total number of successes in n independent 
trials when the probability of success for each trial is p. 
The Poisson distribution P(r) with 


x 


T 
(2) P(X=x)= Te", x cnr d eT. 
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This is the distribution of the number of events occurring in a fixed interval 
of time or space if the probability of more than one occurrence in a very 
short interval is of smaller order of magnitude than that of a single 
occurrence, and if the numbers of events in nonoverlapping intervals are 
statistically independent. Under these assumptions, the process generating 
the events is called a Poisson process. Such processes are discussed, for 
example, in the books by Feller (1968), Karlin and Taylor (1975), and Ross 
(1980). 
The normal distribution N(£, o?) with probability density 


1 


1 
O px) =acew|-galx- S) cm ERR nn 


Under very general conditions, which are made precise by the central limit 
theorem, this is the approximate distribution of the sum of a large number 
of independent random variables when the relative contribution of each 


term to the sum is small. 
We consider next the.structure of the decision space D. The great variety 
of possibilities is indicated by the following examples. 


Example 1. Let X,,..., X, be a sample from one of the distributions (1)-(3), 
that is, let the X’s be distributed independently and identically according to one of 
these distributions. Let @ be p, 7, or the pair (£, o) respectively, and let y = y(9) 
be a real-valued function of 6. 


(i If one wishes to decide whether or not y exceeds some specified value Yo, the 
choice lies between the two decisions dg: y » Yo and di: Y < Yo- In specific 
applications these decisions might correspond to the acceptance or rejection of a lot 
of manufactured goods, of an experimental airplane as ready for flight testing, of a 
new treatment as an improvement over a standard one, and so on. The loss function 
of course depends on the application to be made. Typically, the loss is 0 if the 
correct decision is chosen, while for an incorrect decision the losses L(y, do) and 
L(y, d,) are increasing functions of |y — Yol- 

(ii) At the other end of the scale is the much more detailed problem of 
obtaining a numerical estimate of y. Here a decision d of the statistician is a real 
number, the estimate of y, and the losses might be L(y, d) = v(y)w(ld — yD, 
where w is a strictly increasing function of the error |d — yl. 

(iii) An intermediate case is the choice between the three alternatives do : Y < Yo» 
di: Y> Y doi Yo S Y S Y for example accepting a new treatment, rejecting it, or 
recommending it for further study. 


The distinction illustrated by this example is the basis for one of the 
principal classifications of statistical methods. Two-decision problems such 
as (i) are usually formulated in terms of testing a hypothesis which is to be 
accepted or rejected (see Chapter 3). It is the theory of this class of problems 
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with which we shall be mainly concerned here. The other principal branch 
of statistics is the theory of point estimation dealing with problems such as 
(ii). This is the subject of TPE. The intermediate problem (iii) is a special 
case of a multiple decision procedure. Some problems of this kind are treated 
in Ferguson (1967, Chapter 6); a discussion of some others is given in 
Chapter 7, Section 4. 


Example 2. Suppose that the data consist of samples Xj. J= Ll... nj, from 
normal populations N(£,, 0°), i= 1,..., s. 


(i) Consider first the case s — 2 and the question of whether or not there is a 
material difference between the two populations. This has the same structure as 
problem (iii) of the previous example. Here the choice lies between the three 
decisions do:|£; — &| < A, di : $z > & + A, d3: §, < £f, — A, where A is pre- 
assigned. An analogous problem, involving k + 1 possible decisions, occurs in the 
general case of k populations. In this case one must choose between the decision 
that the k distributions do not differ materially, dọ : max|£, — £&,| < A, and the 
decisions d, : max|£, — | > A and £, is the largest of the means. 

(ii) A related problem is that of ranking the distributions in increasing order of 
their mean £. 

(iii) Alternatively, a standard £y may be given and the problem is to decide 
which, if any, of the population means exceed the standard. 


Example 3. Consider two distributions—to be specific, two Poisson distribu- 
tions P(7,), P(r;)—and suppose that 7, is known to be less than 7, but that 
otherwise the 7's are unknown. Let Z;,..., Z, be independently distributed, each 
according to either P(1,) or P(;). Then each Z is to be classified as to which of the 
two distributions it comes from. Here the loss might be the number of Z's that are 
incorrectly classified, multiplied by a suitable function of 7, and 7,. An example of 
the complexity that such problems can attain and the conceptual as well as 
mathematical difficulties that they may involve is provided by the efforts of 
anthropologists to classify the human population into a number of homogeneous 


races by studying the frequencies of the various blood groups and of other genetic 
characters. 


All the problems considered so far could be termed action problems. lt 
was assumed in all of them that if @ were known a unique correct decision 
would be available, that is, given any 6, there exists a unique d for which 
L(0, d) = 0. However, not all statistical problems are so clear-cut. Fre- 
quently it is a question of providing a convenient summary of the data or 
indicating what information is available concerning the unknown parameter 
or distribution. This information will be used for guidance in various 
considerations but will not provide the sole basis for any specific decisions. 
In such cases the emphasis is on the inference rather than on the decision 
aspect of the problem. Although formally it can still be considered a 
decision problem if the inferential statement itself is interpreted as the 
decision to be taken, the distinction is of conceptual and practical signifi- 


1.2] SPECIFICATION OF A DECISION PROBLEM 5 


cance despite the fact that frequently it is ignored.* An important class of 
such problems, estimation by interval, is illustrated by the following exam- 
ple. (For the more usual formulation in terms of confidence intervals, see 
Chapter 3, Section 5, and Chapter 5, Sections 4 and 5.) 


Example 4. Let X =(X,,.--) Xn) be a sample from N(£, o?) and let a decision 
consist in selecting an interval [L, L] and stating that it contains $. Suppose that 


decision procedures are restricted to intervals [L( X), L( X)] whose expected length 
for all £ and o does not exceed ko where k is some preassigned constant. An 
appropriate loss function would be 0 if the decision is correct and would otherwise 
depend on the relative position of the interval to the true value of ¢. In this case 
there are many correct decisions corresponding to a given distribution N(£, 0”). 


It remains to discuss the choice of loss function,’ and of the three 
elements defining the problem this is perhaps the most difficult to specify. 
Even in the simplest case, where all losses eventually reduce to financial 
ones, it can hardly be expected that one will be able to evaluate all the 
short- and long-term consequences of an action. Frequently it is possible to 
simplify the formulation by taking into account only certain aspects of the 
loss function. As an illustration consider Example 1(i) and let L(8, do) = a 
for y(0) < Yo and L(9, d,) = b for y(0) > Yo- The risk function becomes 


" aP,(8(X)- dy) if YSYo 
(4) (0,8) = (Orta a) - 4) if y? Yo 


and is seen to involve only the two probabilities of error, with weights which 
can be adjusted according to the relative importance of these errors. 
Similarly, in Example 3 one may wish to restrict attention to the number of 
misclassifications. 

Unfortunately, such a natural simplification is not always available, and 
in the absence of specific knowledge it becomes necessary to select the loss 
function in some conventional way, with mathematical simplicity usually an 
important consideration. In point estimation problems such as that consid- 
ered in Example l(i) if one is interested in estimating a real-valued 
function y = y(8) it is customary to take the square of the error, or 
somewhat more generally to put 


(5) L(0,d) = v(6)(d — yy. 


*For a more detailed discussion of this distinction see, for example, Cox (1958), Blyth 


(1970), and Barnett (1982). - 
fSome aspects of the choice of model and loss function are discussed in Lehmann (1984, 


1985). 
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Besides being particularly simple mathematically, this can be considered as 
an approximation to the true loss function L provided that for each fixed 6, 
L(6, d) is twice differentiable in d, that L(0, y(@)) = 0 for all 0, and that 
the error is not large. 

It is frequently found that, within one problem, quite different types of 
losses may occur, which are difficult to measure on a common scale. 
Consider once more Example 1(i) and suppose that y; is the value of y 
when a standard treatment is applied to a situation in medicine, agriculture, 
or industry. The problem is that of comparing some new process with 
unknown y to the standard one. Turning down the new method when it is 
actually superior, or adopting it when it is not, clearly entails quite different 
consequences. In such cases it is sometimes convenient to treat the various 
loss components, say Li, L»,..., L,, separately. Suppose in particular that 
r= 2 and that L, represents the more serious possibility. One can then 
assign a bound to this risk component, that is, impose the condition 


(6) EL,(0,9(X)) < a, 


and subject to this condition minimize the other component of the risk. 
Example 4 provides an illustration of this procedure. The length of the 
interval [ L, L] (measured in ø-units) is one component of the loss function, 
the other being the loss that results if the interval does not cover the true £ 


3. RANDOMIZATION; CHOICE OF EXPERIMENT 


The description of the general decision problem given so far is still too 
narrow in certain respects. It has been assumed that for each possible value 
of the random variables a definite decision must be chosen. Instead, it is 
convenient to permit the selection of one out of a number of decisions 
according to stated probabilities, or more generally the selection of a 
decision according to a probability distribution defined over the decision 
space; which distribution depends of course on what x is observed. One 
way to describe such a randomized procedure is in terms of a nonran- 
domized procedure depending on X and a random variable Y whose values 
lie in the decision space and whose conditional distribution given x is 
independent of 0. 

Although it may run counter to one's intuition that such extra randomi- 
zation should have any value, there is no harm in permitting this greater 
freedom of choice. If the intuitive misgivings are correct, it will turn out that 
the optimum procedures always are of the simple nonrandomized kind. 
Actually, the introduction of randomized procedures leads to an important 
mathematical simplification by enlarging the class of risk functions so that it 
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becomes convex. In addition, there are problems in which some features of 
the risk function such as its maximum can be improved by using a 
randomized procedure. 

Another assumption that tacitly has been made so far is that a definite 
experiment has already been decided upon so that it is known what 
observations will be taken. However, the statistical considerations involved 
in designing an experiment are no less important than those concerning its 
analysis. One question in particular that must be decided before an investi- 
gation is undertaken is how many observations should be taken so that the 
risk resulting from wrong decisions will not be excessive. Frequently it turns 
out that the required sample size depends on the unknown distribution and 
therefore cannot be determined in advance as a fixed number. Instead it is 
then specified as a function of the observations and the decision whether or 
not to continue experimentation is made sequentially at each stage of the 
experiment on the basis of the observations taken up to that point. 


Example 5. On the basis of a sample X,,..., X, from a normal distribution 
N(£, o?) one wishes to estimate £. Here the risk function of an estimate, for 
example its expected squared error, depends on c. For large c the sample contains 
only little information in the sense that two distributions N(£, o?) and N(£;, o?) 
with fixed difference £, — £, become indistinguishable as o > oo, with the result 
that the risk tends to infinity. Conversely, the risk approaches zero as a > 0, since 
then effectively the mean becomes known. Thus the number of observations needed 
to control the risk at a given level is unknown. However, as soon as some 
observations have been taken, it is possible to estimate c? and hence to determine 
the additional number of observations required. 


Example 6. In a sequence of trials with constant probability p of success, one 
wishes to decide whether p « 3 or p > 1. It will usually be possible to reach a 
decision at an early stage if p is close to 0 or 1 so that practically all observations 
are of one kind, while a larger sample will be needed for intermediate values of p. 


loss resulting from a wrong decision is presumably less serious than for the more 
extreme values. 


Example 7. The possibility of determining the sample size sequentially is 
important not only because the distributions P, can be more or less informative but 
also because the same is true of the observations themselves. Consider, for example, 
observations from the uniform distribution over the interval (0 — 1, @ + 1) and the 
problem of estimating 0. Here there is no difference in the amount of information 
provided by the different distributions Pj. However, a sample X,, X,,..., X, can 
practically pinpoint @ if max|X, — X;| is sufficiently close to 1, or it can give 
essentially no more information a single observation if max|X, — X;| is close to 
0. Again the required sample size should be determined sequentially. 


Except in the simplest situations, the determination of the appropriate 
sample size is only one aspect of the design problem. In general, one must 
decide not only how many but also what kind of observations to take. In 
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clinical trials, for example, when a new treatment is being compared with a 
standard procedure, a protocol is required which specifies to which of the 
two treatments each of the successive incoming patients is to be assigned. 
Formally, such questions can be subsumed under the general decision 
problem described at the beginning of the chapter, by interpreting X as the 
set of all available variables, by introducing the decisions whether or not to 
stop experimentation at the various stages, by specifying in case of con- 
tinuance which type of variable to observe next, and by including the cost of 
observation in the loss function. 

The determination of optimum sequential stopping rules and experimen- 
tal designs is outside the scope of this book. Introductions to these subjects 
are provided, for example, by Chernoff (1972) Ghosh (1970) and 
Govindarajulu (1981). 


4. OPTIMUM PROCEDURES 


At the end of Section 1 the aim of statistical theory was stated to be the 
determination of a decision function à which minimizes the risk function 


(7) R(8,8) = E,[L(6, 8(X))]. 


Unfortunately, in general the minimizing ô depends on 0, which is 
unknown. Consider, for example, some particular decision dọ, and the 
decision procedure 5(x) = dy according to which decision dy is taken 
regardless of the outcome of the experiment. Suppose that d, is the correct 
decision for some 6,, so that L(&,, dọ) = 0. Then à minimizes the risk at 6, 
Ta R(, 8) = 0, but presumably at the cost of a high risk for other values 
of 0. 

In the absence of a decision function that minimizes the risk for all 6, the 
mathematical problem is still not defined, since it is not clear what is meant 
by a best procedure. Although it does not seem possible to give a definition 
of optimality that will be appropriate in all situations, the following two 
methods of approach frequently are satisfactory. 

The nonexistence of an optimum decision rule is a consequence of the 
possibility that a procedure devotes too much of its attention to a single 
parameter value at the cost of neglecting the various other values that might 
arise. This suggests the restriction to decision procedures which possess a 
certain degree of impartiality, and the possibility that within such a re- 
stricted class there may exist a procedure with uniformly smallest risk. Two 
conditions of this kind, invariance and unbiasedness, will be discussed in 
the next section. 
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Figure 1 


Instead of restricting the class of procedures, one can approach the 
problem somewhat differently. Consider the risk functions corresponding to 
two different decision rules 6, and 4,. If R(0,8,) < R(6, 65) for all 6, then 
5, is clearly preferable to ô, since its use will lead to a smaller risk no 
matter what the true value of @ is. However, the situation is not clear when 
the two risk functions intersect as in Figure 1. What is needed is a principle 
which in such cases establishes a preference of one of the two risk functions 
over the other, that is, which introduces an ordering into the set of all risk 
functions. A procedure will then be optimum if its risk function is best 
according to this ordering. Some criteria that have been suggested for 
ordering risk functions will be discussed in Section 6. 

A weakness of the theory of optimum procedures sketched above is its 
dependence on an extraneous restricting or ordering principle, and on 
knowledge concerning the loss function and the distributions of the observ- 
able random variables which in applications is frequently unavailable or 
unreliable. These difficulties, which may raise doubt concerning the value of 
an optimum theory resting on such shaky foundations, are in principle no 
different from those arising in any application of mathematics to reality. 
Mathematical formulations always involve simplification and approxima- 
tion, so that solutions obtained through their use cannot be relied upon 
without additional checking. In the present case à check consists in an 
overall evaluation of the performance of the procedure that the theory 
produces, and an investigation of its sensitivity to departure from the 
assumptions under which it was derived. 

The optimum theory discussed in this book should therefore not be 
understood to be prescriptive. The fact that a procedure 8 is optimal 
according to some optimality criterion does not necessarily mean that it is 
the right procedure to use, or even a satisfactory procedure. It does show 
how well one can do in this particular direction and how much is lost when 
other aspects have to be taken into account. 
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The aspect of the formulation that typically has the greatest influence on 
the solution of the optimality problem is the family P to which the 
distribution of the observations is assumed to belong. The investigation of 
the robustness of a proposed procedure to departures from the specified 
model is an indispensable feature of a suitable statistical procedure, and 
although optimality (exact or asymptotic) may provide a good starting 
point, modifications are often necessary before an acceptable solution is 
found. It is possible to extend the decision-theoretic framework to include 
robustness as well as optimality. Suppose robustness is desired against some 
class 2’ of distributions which is larger (possibly much larger) than the 
given #, Then one may assign a bound M to the risk to be tolerated over 
9'. Within the class of procedures satisfying this restriction, one can then 
optimize the risk over P as before. Such an approach has been proposed 
and applied to a number of specific problems by Bickel (1984). 

Another possible extension concerns the actual choice of the family 2, 
the model used to represent the actual physical situation. The problem of 
choosing a model which provides an adequate description of the situation 
without being unnecessarily complex can be treated within the decision- 
theoretic formulation of Section 1 by adding to the loss function a compo- 
nent representing the complexity of the proposed model. For a discussion of 
such an approach to model selection, see Stone (1981). 


5. INVARIANCE AND UNBIASEDNESS* 


A. natural definition of impartiality suggests itself in situations which are 
symmetric with respect to the various parameter values of interest: The 
procedure is then required to act symmetrically with respect to these values. 


Example 8. Suppose two treatments are to be compared and that each is 
applied n times. The resulting observations Xj;,..., Xj, and X5,..., Xj, are 
samples from N(£,, o?) and N(£;, o?) respectively. The three available decisions 
are di lf — &| < A, dif » & +A, d3: £z < & — A, and the loss is wj if 
decision d, is taken when d; would have been correct. If the treatments are to be 
compared solely in terms of the £'s and no outside considerations are involved, the 
losses are symmetric with respect to the two treatments so that Wy, = Wo2, Wio = W20» 
wi; = wy. Suppose now that the labeling of the two treatments as 1 and 2 is 
reversed, and correspondingly also the labeling of the X’s, the £'s, and the decisions 
d, and d;. This changes the meaning of the symbols, but the formal decision 
problem, because of its symmetry, remains unaltered. It is then natural to require 
the corresponding symmetry from the procedure à and ask that 8X: Xin? 
Xg- X22) = do, d, OF dj as 8x3... X2, Xii) = do, d3, or d 
respectively. If this condition were not satisfied, the decision as to which population 


*The concepts discussed here for general decision theory will be developed in more 
specialized form in later chapters. The present section may therefore be omitted at first reading. 
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has the greater mean would depend on the presumably quite accidental and 
irrelevant labeling of the samples. Similar remarks apply to a number of further 
symmetries that are present in this problem. 


Example 9. Consider a sample X,,..., X, from a distribution with density 
o`? f [(x — £)/o] and the problem of estimating the location parameter £, say the 
mean of the X’s, when the loss is (d — £)?/c^, the square of the error expressed in 
c-units. Suppose that the observations are originally expressed in feet, and let 
X/ = aX, with a = 12 be the corresponding observations in inches. In the trans- 
formed problem the density is o' Mf [(x' — £)/6'] with £' = a£, o' = ac. Since 
(d' — £&)!/a? = (d - £)?/o?, the problem is formally unchanged. The same esti- 
mation procedure that is used for the original observations is therefore appropriate 
after the transformation and leads to 8(aX;,..., aX,) as an estimate of £' = a£, the 
parameter £ expressed in inches. On reconverting the estimate into feet one finds 
that if the result is to be independent of the scale of measurements, 5 must satisfy 
the condition of scale invariance 


8(aX,,..., aX,) 
a 


ESL) 


The general mathematical expression of symmetry is invariance under a 
suitable group of transformations. A group G of transformations g of the 
sample space is said to leave a statistical decision problem invariant if it 
satisfies the following conditions: 


(i) It leaves invariant the family of distributions P = (P, 0 € €), that is, 
for any possible distribution P, of X the distribution of gX, say Pp; is 
also in 2. The resulting mapping 0' = g0 of € is assumed to be onto! 
Q and 1:1. 

(ii) To each g € G, there corresponds a transformation g* = h(g) of the 
decision space D onto itself such that h is a homomorphism, that is, 
satisfies the relation A(g;82) ^ ^( g,)h(g2), and the loss function L is 
unchanged under the transformation, so that 


L(g0, g*d) = L(0, d). 


Under these assumptions the transformed problem, in terms of X' — gX, 

' = g0, and d’ = g*d, is formally identical with the original problem in 
terms of X, 0, and d. Given a decision procedure 5 for the latter, this is 
therefore still appropriate after the transformation. Interpreting the trans- 
formation as a change of coordinate system and hence of the names of the 
elements, one would, on observing x’, select the decision which in the new 


f The term onto is used to indicate that Z9 is not only contained in but actually equals Q; 
that is, given any 0’ in Q, there exists 0 in Q such that go - 0'. 
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system has the name (x^), so that its old name is g*- !ó(x^). If the decision 
taken is to be independent of the particular coordinate system adopted, this 
should coincide with the original decision 8(x), that is, the procedure must 
satisfy the invariance condition 


(8) 8(gx) =g*d(x) forall xeX, geG. 


Example 10. The model described in Example 8 is invariant also under the 
transformations X/,= X,, + c, £j =& + c. Since the decisions do, dj, and d, 
concern only the differences $, — £j, they should remain unchanged under these 
transformations, so that one would expect to have g*d, = d, for i = 0,1,2. It is in 
fact easily seen that the loss function does satisfy L(g0, d) = L(0, d), and hence 
that g*d = d. A decision procedure therefore remains invariant in the present case 
if it satisfies 8(gx) = 8(x) for all g E G, x € X. 


It is helpful to make a terminological distinction between situations like 
that of Example 10 in which g*d = d for all d, and those like Examples 8 
and 9 where invariance considerations require 6(gx) to vary with g. In the 
former case the decision procedure remains unchanged under the trans- 
formations X’ = gX and is thus truly invariant; in the latter, the procedure 
varies with g and may then more appropriately be called equivariant rather 
than invariant Typically, hypothesis testing leads to procedures that are 
invariant in this sense; estimation problems (whether by point or interval 
estimation), to equivariant ones. Invariant tests and equivariant confidence 
sets will be discussed in Chapter 6. For a brief discussion of equivariant 
point estimation, see Bondessen (1983); a fuller treatment is given in TPE, 
Chapter 3. 

Invariance considerations are applicable only when a problem exhibits 
certain symmetries. An alternative impartiality restriction which is appli- 
cable to other types of problems is the following condition of unbiasedness. 
Suppose the problem is such that for each 6 there exists a unique correct 
decision and that each decision is correct for some 0. Assume further that 
L(0,, d) = L(0,, d) for all d whenever the same decision is correct for 
both 6, and @,. Then the loss L(0, d^) depends only on the actual decision 
taken, say d', and the correct decision d. The loss can thus be denoted by 
L(d, d") and this function measures how far apart d and d' are. Under 
these assumptions a decision function 6 is said to be unbiased with respect 
to the loss function L, or L-unbiased, if for all @ and d’ 


E,L(d',8(X)) 2 E,L(d, 8(X)) 


where the subscript @ indicates the distribution with respect to which the 


* This distinction is not adopted by all authors. 


1.5] INVARIANCE AND UNBIASEDNESS 13 


expectation is taken and where d is the decision that is correct for 6. Thus à 
is unbiased if on the average ô( X) comes closer to the correct decision than 
to any wrong one. Extending this definition, ô is said to be L-unbiased for 
an arbitrary decision problem if for all à and 6’ 


(9) E,L(6’,8(X)) = EL (0, 6(X)). 


Example 11. Suppose that in the problem of estimating a real-valued parameter 
0 by confidence intervals, as in Example 4, the loss is 0 or 1 as the interval [ L, L] 
does or does not cover the true 0. Then the set of intervals [L( X), L(X)] is 
unbiased if the probability of covering the true value is greater than or equal to the 
probability of covering any false value. 


Example 12. In a two-decision problem such as that of Example 1(i), let wọ and 
w, be the sets of @-values for which dy and d, are the correct decisions. Assume 
that the loss is 0 when the correct decision is taken, and otherwise is given by 
L(0, do) = a for 0 € w, and L(0, d,) = b for 0 € «y. Then 


Se Pine 1522 
so that (9) reduces to 
aP,(8(X) = do} = bPa {8(X) = 4) for 0€ oy, 
with the reverse inequality holding for 0 € w,. Since P,(6(X) = dy} + Py{ 8(X) 
= d,} = 1, the unbiasedness condition (9) becomes 
a 


P,{ 8(X) = d) SIE. for 0 Ew, 


(10) 4 
B(sX)-4)z75 for 0 Ew. 


Example 13. In the problem of estimating a real-valued function y(0) with the 
square of the error as loss, the condition of unbiasedness becomes 


E8(X) - (0)? > E[8(X) - (0)? for all 0,0". 


On adding and subtracting (0) = E,9(X) inside the brackets on both sides, this 
reduces to 


LAÇO) - (6)P = [4(0) 7 Y(D. forall 6, 0". 


If h(8) is one of the possible values of the function y, this condition holds if and 
only if 


(11) E&(X) = (0). 
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In the theory of point estimation, (11) is customarily taken as the definition of 
unbiasedness. Except under rather pathological conditions, it is both a necessary 
and sufficient condition for 8 to satisfy (9). (See Problem 2.) 


6. BAYES AND MINIMAX PROCEDURES 


We now turn to a discussion of some preference orderings of decision 
procedures and their risk functions. One such ordering is obtained by 
assuming that in repeated experiments the parameter itself is a random 
variable @, the distribution of which is known. If for the sake of simplicity 
one supposes that this distribution has a probability density p(8), the 
overall average loss resulting from the use of a decision procedure à is 


Q2) — r(p,8) = f EL (0, 50)0(0) dð = [R(0, 8)0(8) 40 


and the smaller r(p, 8), the better is 6. An optimum procedure is one that 
minimizes r(p,6) and is called a Bayes solution of the given decision 
problem corresponding to the a priori density p. The resulting minimum of 
r(p, 5) is called the Bayes risk of ô. 

Unfortunately, in order to apply this principle it is necessary to assume 
not only that 6 is a random variable but also that its distribution is known. 
This assumption is usually not warranted in applications. Alternatively, the 
right-hand side of (12) can be considered as a weighted average of the risks; 
for p(0) = 1 in particular, it is then the area under the risk curve. With this 
interpretation the choice of a weight function p expresses the importance 
the experimenter attaches to the various values of 0. A systematic Bayes 
theory has been developed which interprets p as describing the state of 
mind of the investigator towards 0. For an account of this approach see, for 
example, Berger (1985). 

If no prior information regarding @ is available, one might consider the 
maximum of the risk function its most important feature. Of two risk 
functions the one with the smaller maximum is then preferable, and the 
optimum procedures are those with the minimax property of minimizing the 
maximum risk. Since this maximum represents the worst (average) loss that 
can result from the use of a given procedure, a minimax solution is one that 
gives the greatest possible protection against large losses. That such a 
principle may sometimes be quite unreasonable is indicated in Figure 25 
where under most circumstances one would prefer ô, to ô, although its risk 
function has the larger maximum. 

Perhaps the most common situation is one intermediate to the two just 
described. On the one hand, past experience with the same or similar kind 
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Figure 2 


of experiment is available and provides an indication of what values of @ to 
expect; on the other, this information is neither sufficiently precise nor 
sufficiently reliable to warrant the assumptions that the Bayes approach 
requires. In such circumstances it seems desirable to make use of the 
available information without trusting it to such an extent that catastrophi- 
cally high risks might result if it is inaccurate or misleading. To achieve this 
one can place a bound on the risk and restrict consideration to decision 
procedures ô for which 


(13) R(0,8)<C forall @. 


[Here the constant C will have to be larger than the maximum risk C, of the 
minimax procedure, since otherwise there will exist no procedures satisfying 
(13).] Having thus assured that the risk can under no circumstances get out 
of hand, the experimenter can now safely exploit his knowledge of the 
situation, which may be based on theoretical considerations as well as on 
past experience; he can follow his hunches and guess at a distribution p for 
0. This leads to the selection of a procedure 6 (a restricted Bayes solution), 
which minimizes the average risk (12) for this a priori distribution subject to 
(13). The more certain one is of p, the larger one will select C, thereby 
running a greater risk in case of a poor guess but improving the risk if the 
guess is good. 

Instead of specifying an ordering directly, one can postulate conditions 
that the ordering should satisfy. Various systems of such conditions have 
been investigated and have generally led to the conclusion that the only 
orderings satisfying these systems are those which order the procedures 
according to their Bayes risk with respect to some prior distribution of 6. 
For details, see for example Blackwell and Girshick (1954), Ferguson (1967), 
Savage (1972), and Berger (1985). 
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7. MAXIMUM LIKELIHOOD 


Another approach, which is based on considerations somewhat different 
from those of the preceding sections, is the method of maximum likelihood. 
It has led to reasonable procedures in a great variety of problems, and is 
still playing a dominant role in the development of new tests and estimates. 
Suppose for a moment that X can take on only a countable set of values 
Xj, X2. .., With Py(x) = P){ X = x}, and that one wishes to determine the 
correct value of 6, that is, the value that produced the observed x. This 
suggests considering for each possible 8 how probable the observed x 
would be if @ were the true value. The higher this probability, the more one 
is attracted to the explanation that the 0 in question produced x, and the 
more likely the value of 0 appears. Therefore, the expression P,(x) consid- 
ered for fixed x as a function of @ has been called the likelihood of 0. To 
indicate the change in point of view, let it be denoted by L,(0). Suppose 
now that one is concerned with an action problem involving a countable 
number of decisions, and that it is formulated in terms of a gain function 
(instead of the usual loss function), which is 0 if the decision taken is 
incorrect and is a(@) > 0 if the decision taken is correct and @ is the true 
value. Then it seems natural to weight the likelihood L,(@) by the amount 
that can be gained if @ is true, to determine the value of ð that maximizes 
a(@)L,(@) and to select the decision that would be correct if this were the 
true value of 0. Essentially the same remarks apply in the case in which 
P,(x) is a probability density rather than a discrete probability. 

In problems of point estimation, one usually assumes that a(@) is 
independent of @. This leads to estimating @ by the value that maximizes the 
likelihood L,(@), the maximum-likelihood estimate of 6. Another case of 
interest is the class of two-decision problems illustrated by Example 1(i). Let 
w and w, denote the sets of -values for which dy and d, are the correct 
decisions, and assume that a(0) = ay or a, as 0 belongs to wy or w, 
respectively. Then decision dy or d, is taken as ajsups c, L,(0) < or 
- 498Up e ,,L.,(0), that is, as 


sup L,(6) 

be a 
14 Fi, RE as 3 
ke UID L0. ae sidus 

6€o, 


This is known as a likelihood-ratio procedure.* 


*This definition differs slightly from the usual one where in the denominator on the 
left-hand side of (14) the supremum is taken over the set a; U w. The two definitions agree 
whenever the left-hand side of (14) is < 1, and the procedures therefore agree if a, < aj. 
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Although the maximum-likelihood principle is not based on any clearly 
defined optimum considerations, it has been very successful in leading to 
satisfactory procedures in many specific problems. For wide classes of 
problems, maximum-likelihood procedures have also been shown to possess 
various asymptotic optimum properties as the sample size tends to infinity. 
[An asymptotic theory of likelihood-ratio tests has been developed by Wald 
(1943) and Le Cam (1953, 1979); an overview with additional references is 
given by Cox and Hinkley (1974). The corresponding theory of maximum- 
likelihood estimators is treated in Chapter 6 of TPE.] On the other hand, 
there exist examples for which the maximum-likelihood procedure is worse 
than useless; where it is, in fact, so bad that one can do better without 
making any use of the observations (see Chapter 6, Problem 18). 


8. COMPLETE CLASSES 


None of the approaches described so far is reliable in the sense that the 
resulting procedure is necessarily satisfactory. There are problems in which 
a decision procedure à exists with uniformly minimum risk among all 
unbiased or invariant procedures, but where there exists a procedure 6, not 
possessing this particular impartiality property and preferable to 4). (Cf. 
Problems 14 and 16.) As was seen earlier, minimax procedures can also be 
quite undesirable, while the success of Bayes and restricted Bayes solutions 
depends on a priori information which is usually not very reliable if it is 
available at all. In fact, it seems that in the absence of reliable a priori 
information no principle leading to a unique solution can be entirely 
satisfactory. 

This suggests the possibility, at least as a first step, of not-insisting on a 
unique solution but asking only how far a decision problem can be reduced 
without loss of relevant information. It has already been seen that a decision 
procedure à can sometimes be eliminated from consideration because there 
exists a procedure 6’ dominating it in the sense that 


R(8,8) < R(6,8) forall @ 
(15) 
R(6,8’) < R(0,8) for some 0. 


In this case ô is said to be inadmissible; à is called admissible if no such 
dominating 8’ exists. A class € of decision procedures is said to be complete 
if for any 6 not in € there exists 6’ in € dominating it. A complete class is 
minimal if it does not contain a complete subclass. If a minimal complete 
class exists, as is typically the case, it consists exactly of the totality of 
admissible procedures. 
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It is convenient to define also the following variant of the complete class 
notion. A class € is said to be essentially complete if for any procedure ô 
there exists 6’ in € such that R(0, 5’) < R(0, 8) for all 0. Clearly, any 
complete class is also essentially complete. In fact, the two definitions differ 
only in their treatment of equivalent decision rules, that is, decision rules 
with identical risk function. If ô belongs to the minimal complete class @, 
any equivalent decision rule must also belong to @. On the other hand, a 
minimal essentially complete class need contain only one member from such 
a set of equivalent procedures. 

In a certain sense a minimal essentially complete class provides the 
maximum possible reduction of a decision problem. On the one hand, there 
is no reason to consider any of the procedures that have been weeded out. 
For each of them, there is included one in € that is as good or better. On 
the other hand, it is not possible to reduce the class further. Given any two 
procedures in @, each of them is better in places than the other, so that 
without additional information it is not known which of the two is prefer- 
able. 

The primary concern in statistics has been with the explicit determination 
of procedures, or classes of procedures, for various specific decision prob- 
lems. Those studied most extensively have been estimation problems, and 
problems involving a choice between only two decisions (hypothesis testing), 
the theory of which constitutes the subject of the present volume. However, 
certain conclusions are possible without such specialization. In particular, 
two results concerning the structure of complete classes and minimax 
procedures have been proved to hold under very general assumptions:* 


© The totality of Bayes solutions and limits of Bayes solutions con- 
stitute a complete class. 

(ii) Minimax procedures are Bayes solutions with respect to a least 
favorable a priori distribution, that is, an a priori distribution that maxi- 
mizes the associated Bayes risk, and the minimax risk equals this maximum 
Bayes risk. Somewhat more generally, if there exists no least favorable 
a priori distribution but only a sequence for which the Bayes risk tends to 
the maximum, the minimax procedures are limits of the associated sequence 
of Bayes solutions. 


9. SUFFICIENT STATISTICS 


A minimal complete class was seen in the preceding section to provide the 
maximum possible reduction of a decision problem without loss of informa- 


*Precise statements and proofs of these results are given in the book by Wald (1950). See 
also Ferguson (1967) and Berger (1985). 
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tion. Frequently it is possible to obtain a less extensive reduction of the 
data, which applies simultaneously to all problems relating to a given class 
P={P, 0 € Q) of distributions of the given random variable X. It 
consists essentially in discarding that part of the data which contains no 
information regarding the unknown distribution Py, and which is therefore 
of no value for any decision problem concerning 6. 


Example 14. Trials are performed with constant unknown probability p of 
success. If X; is 1 or 0 as the ith trial is a success or failure, the sample (X;,..., X,) 
shows how many successes there were and in which trials they occurred. The second 
of these pieces of information contains no evidence as to the value of p. Once the 
total number of successes EX; is known to be equal to t, each of the ") possible 
positions of these successes is equally likely regardless of p. It follows that knowing 
E X, but neither the individual X, nor p, one can, from a table of random numbers, 
construct a set of random variables X/,..., X; whose joint distribution is the same 
as that of X,,..., X,. Therefore, the information contained in the X, is the same as 
that contained in XX, and a table of random numbers. 


Example 15. If X,,..., X, are independently normally distributed with zero 
mean and variance o?, the conditional distribution of the sample point over each of 
the spheres, =X? = constant, is uniform irrespective of c?. One can therefore 
construct an equivalent sample Xj,..., X; from a knowledge of ZX? and a 
mechanism that can produce a point randomly distributed over a sphere. 


More generally, a statistic T is said to be sufficient for the family 
P= ( P,, 0 € €) (or sufficient for 6, if it is clear from the context what set 
Q is being considered) if the conditional distribution of X given T = t is 
independent of 0. As in the two examples it then follows under mild 
assumptions* that it is not necessary to utilize the original observations X. 
If one is permitted to observe only T instead of X, this does not restrict the 
class of available decision procedures. For any value t of T let X, bea 
random variable possessing the conditional distribution of X given t. Such a 
variable can, at least theoretically, be constructed by means of a suitable 
random mechanism. If one then observes T to be t and X, to be x’, the 
random variable X' defined through this two-stage process has the same 
distribution as X. Thus, given any procedure based on X, it is possible to 
construct an equivalent one based on X' which can be viewed as a 
randomized procedure based solely on T. Hence if randomization is per- 
mitted (and we shall assume throughout that this is the case), there is no loss 
of generality in restricting consideration to a sufficient statistic. 

It is inconvenient to have to compute the conditional distribution of X 
given ¢ in order to determine whether or not T is sufficient. A simple check 
is provided by the following factorization criterion. 


"These are connected with difficulties concerning the behavior of conditional probabilities. 
For a discussion of these difficulties see Chapter 2, Sections 3-5. 
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Consider first the case that X is discrete, and let P,(x) = Po{ X = x). 
Then a necessary and sufficient condition for T to be sufficient for @ is that 
there exists a factorization 


(16) P(x) = golT(x)] A(x), 


where the first factor may depend on @ but depends on x only through 
T(x), while the second factor is independent of 0. 

Suppose that (16) holds, and let T(x) = t. Then P,{T =t} = LP, (x’) 
summed over all points x’ with T(x’) = t, and the conditional probability 


P,(x) h(x) 
(X= x70 Fre) ERG 


is independent of @. Conversely, if this conditional distribution does not 
depend on @ and is equal to, say k(x, t), then P(x) = Po{T = t) k(x, t), 
so that (16) holds. 


Example 16. Let X,..., X, be independently and identically distributed 
according to the Poisson distribution (2). Then 


Erie ™ 


LAG Hee AA 3 


TI x;! 


j=l 


and it follows that X X, is a sufficient statistic for 7. 

In the case that the distribution of X is continuous and has probability 
density p(x), let X and T be vector-valued, X =(X,..., Xn) and 
T = (T,,..., T) say. Suppose that there exist functions Y = (Yy,..., Yn-r) 
on the sample space such that the transformation 


Q7) (rxn) e (T(x), T(x), G0, 7,0) 


is 1:1 on a suitable domain, and that the joint density of T and Y exists 
and is related to that of X by the usual formula 


(18) pi(x) = pi (T(x) ¥(x)) - MI 

where J is the Jacobian of (T... T, Yy..., Y, ,) with respect to 
(X3, ++) X,). Thus in Example 15, T = (EX? , Y,,..., Y,., can be taken to 
be the polar coordinates of the sample point. From the joint density 


pi: (t, y) of T and Y, the conditional density of Y given T — t is obtained 
as 


9) "ac en a 


fers») oy 
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provided the denominator is different from zero. Regularity conditions for 
the validity of (18) are given by Tukey (1958). 

Since in the conditional distribution given t only the Y's vary, T is 
sufficient for 0 if the conditional distribution of Y given t is independent of 
8. Suppose that T satisfies (19). Then analogously to the discrete case, a 
necessary and sufficient condition for T to be sufficient is a factorization of 
the density of the form 


(20) pé(x) = golT(x)] A(x). 


(See Problem 19.) The following two examples illustrate the application of 
the criterion in this case. In both examples the existence of functions Y 
satisfying (17)-(19) will be assumed but not proved. As will be shown later 
(Chapter 2, Section 6), this assumption is actually not needed for the 
validity of the factorization criterion. 


Example 17. Let X,,..., X, be independently distributed with normal prob- 
ability density 


^ 1 $ n 
ps 2\72/2 poten 2- giles pay) 
Pe. o(*) (20 ) ex 202 Lx Sk o? Lx, 35 ) 


Then the factorization criterion shows (EX EX?) to be sufficient for ($, o). 


Example 18 Let X,,..., X, be independently distributed according to the 
uniform distribution U(0, 0) over the interval (0, 0). Then p(x) = 07" u(max x;, 0), 
where u(a, b) is 1 or 0 as a < b or a> b, and hence max X, is sufficient for 9. 

An alternative criterion of Bayes sufficiency, due to Kolmogorov (1942), 
provides a direct connection between this concept and some of the basic 
notions of decision theory. As in the theory of Bayes solutions, consider the 
unknown parameter 0 as a random variable © with an a priori distribution, 
and assume for simplicity that it has a density p(0). Then if T is sufficient, 
the conditional distribution of @ given X = x depends only on T(x). 
Conversely, if p(@) + 0 for all @ and if the conditional distribution of © 
given x depends only on T(x), then T is sufficient for 0. 

In fact, under the assumptions made, the joint density of X and @ is 
po(x)p(0). If T is sufficient, it follows from (20) that the conditional density 
of © given x depends only on T(x). Suppose, on the other hand, that for 
some a priori distribution for which p(0) #0 for all @ the conditional 
distribution of © given x depends only on T(x). Then 


py(x)e(8) 
[oo(x)o(0') at 


and by solving for p,(x) it is seen that T is sufficient. 


= fe[T(x)] 


B.C.B.R.Y , West senge 
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Any Bayes solution depends only on the conditional distribution of © 
given x (see Problem 8) and hence on T(x). Since typically Bayes solutions 
together with their limits form an essentially complete class, it follows that 
this is also true of the decision procedures based on T. The same conclusion 
had already been reached more directly at the beginning of the section. 

For a discussion of the relation of these different aspects of sufficiency in 
more general circumstances and references to the literature see Le Cam 
(1964) and Roy and Ramamoorthi (1979). An example of a statistic which is 
Bayes sufficient in the Kolmogorov sense but not according to the definition 
given at the beginning of this section is provided by Blackwell and 
Ramamoorthi (1982). 

By restricting attention to a sufficient statistic, one obtains a reduction of 
the data, and it is then desirable to carry this reduction as far as possible. 
To illustrate the different possibilities, consider once more the binomial 
Example 14. If m is any integer less than n and T, = XX, T;— 
E!’ m+iX then (Tj, T)) constitutes a sufficient statistic, since the condi- 
tional distribution of X,,..., X, given T, = tj, T; = t is independent of p. 
For the same reason, the full sample ( X;,..., X,) itself is also a sufficient 
statistic. However, T = D"_,X; provides a more thorough reduction than 
either of these and than various others that can be constructed. A sufficient 
statistic T is said to be minimal sufficient if the data cannot be reduced 
beyond T without losing sufficiency. For the binomial example in particu- 
lar, E^. X, can be shown to be minimal (Problem 17). This illustrates the 
fact that in specific examples the sufficient statistic determined by inspection 
through the factorization criterion usually turns out to be minimal. Explicit 
procedures for constructing minimal sufficient statistics are discussed in 
Section 1.5 of TPE. 


10. PROBLEMS 
Section 2 


1. The following distributions arise on the basis of assumptions similar to those 
leading to (1)-(3). 


(i) Independent trials with constant probability p of success are carried out 
until a preassigned number m of successes has been obtained. If the 
number of trials required is X + m, then X has the negative binomial 
distribution Nb( p, m): 


P{X=x} Cfo Yn a 50,1,2:.. - 


(ii) In a sequence of random events, the number of events occurring in any 
time interval of length 7 has the Poisson distribution P(Ar), and the 
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numbers of events in nonoverlapping time intervals are independent. 
Then the “waiting time” T, which elapses from the starting point, say 
t = 0, until the first event occurs, has the exponential probability density 


p(t)=Ae™, (20. 


Let Tj, i > 2, be the time elapsing from the occurrence of the (i — 1)st 
event to that of the ith event. Then it is also true, although more difficult 
to prove, that T;, T;,... are identically and independently distributed. A 
proof is given, for example, in Karlin and Taylor (1975). 

(ii) A point X is selected “at random" in the interval (a, b), that is, the 
probability of X falling in any subinterval of (a, b) depends only on the 
length of the subinterval, not on its position. Then X has the uniform 
distribution U(a, b) with probability density 


p(x)=1/(b-a),  a«x«b. 


[Gi): If £ > 0, then T» t if and only if no event occurs in the time interval 
(0, 0] 
Section 5 

Unbiasedness in point estimation. Suppose that y is a continuous real-valued 
function defined over Q which is not constant in any open subset of Q, and 
that the expectation A(0) = Ey5(X) is a continuous function of 0 for every 
estimate 8( X) of y(0). Then (11) is a necessary and sufficient condition for 
8(X) to be unbiased when the loss function is the square of the error. 
[Unbiasedness implies that. y?(0^) — y(0) = 2h(8)y(0^) — y(0)) for all 
0, 6’. If 0 is neither a relative minimum or maximum of y, it follows that there 
exist points 0' arbitrarily close to 0 both such that y(0) + y(0 > and 
< 2h(0), and hence that y(8) = A(0). That this equality also holds for an 
extremum of y follows by continuity, since y is not constant in any open set.] 


Median unbiasedness. 


(i) Areal number m is a median for the random variable Y if P(Y > m d epa, 
P{Y < m) > 1. Then all real a}, a, such that m < a, < a, or m2 aj 
> a, satisfy E|Y — a| € E|Y — a)|. 

(ii) For any estimate 8(X) of y(@), let m- (0) and m*(@) denote the 
infimum and supremum of the medians of 8(X), and suppose that they 
are continuous functions of 9. Let (9) be continuous and not constant 
in any open subset of Q. Then the estimate 8(X) of y(@) is unbiased 
with respect to the loss function L(0, d) = |y(@) — d|if and only if yY(0) 
is a median of 8(X) for each 0. An estimate with this property is said to 
be median-unbiased. 


Nonexistence of unbiased procedures. Let X,,..., X, be independently dis- 
tributed with density (1/2) (( x — £)/a), and let 0 = (£, a). Then no estima- 
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(8) 
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tor of £ exists which is unbiased with respect to the loss function (d — £)*/a*. 
Note. For more general results concerning the nonexistence of unbiased 
procedures see Rojo (1983). 


Let € be any class of procedures that is closed under the transformations of a 
group G in the sense that ô € € implies g'6g'! € € for all g € G. If there 
exists a unique procedure à, that uniformly minimizes the risk within the class 
€, then à, is invariant. If à, is unique only up to sets of measure zero, then it 
is almost invariant, that is, for each g it satisfies the equation &(gx) = g*8(x) 
except on a set N, of measure 0. 


Relation of unbiasedness and invariance. 


() If ô is the unique (up to sets of measure 0) unbiased procedure with 
uniformly minimum risk, it is almost invariant. 

(i) If G is transitive and G* commutative, and if among all invariant 
(almost invariant) procedures there exists a procedure à with uniformly 
minimum risk, then it is unbiased. 

(iii) That conclusion (ii) need not hold without the assumptions concerning 
G* and G is shown by the problem of estimating the mean £ of a normal 
distribution N(£, o?) with loss function (£ — d)?/e?. This remains 
invariant under the groups G, : gx = x + b, —oo < b < oo and G, : gx 
= ax + b,0 « a< co, —oo « b « co. The best invariant estimate rela- 
tive to both groups is X, but there does not exist an estimate which is 
unbiased with respect to the given loss function. 


[(i): This follows from the preceding problem and the fact that when ô is 
unbiased so is g*üg !. 

(ii): It is the defining property of transitivity that given 0, 0^ there exists g 
such that 6’ = g0. Hence for any 0, 0" 


EyL( 8", &(X)) = EgL (80, (X) = EoL (8, g* (X). 
Since G* is commutative, g*~'8, is invariant, so that 


R(0, g*718) > R(8, 8) = EjL(0,8,(X)).l 


Section 6 


Unbiasedness in interval estimation. Confidence intervals I = (L, L) are unbi- 
ased for estimating @ with loss function L(0, I) = (0 - L} + (L— 6)? 
provided E[1(L + L)] = 6 for all 6, that is, provided the midpoint of J is an 
unbiased estimate of @ in the sense of (11). 


+Here and in Problems 6, 7, 11, 15, and 16 the term “invariant” is used in the general sense 
of “invariant or equivariant”. 
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8. Structure of Bayes solutions. 


G) 


(i) 


(iii) 


[G: 


Let © be an unobservable random quantity with probability density 
p (8), and let the probability density of X be p, (x) when © = 0. Then 6 
is a Bayes solution of a given decision problem if for each x the decision 
8(x) is chosen so as to minimize /L(6, 8(x))7(8|x) d0, where 7(8|x) 
= p(0) py(x)/ fp (0) po (x) 40’ is the conditional (a posteriori) probabil- 
ity density of @ given x. 

Let the problem be a two-decision problem with the losses as given in 
Example 12. Then the Bayes solution consists in choosing decision d, if 


aP{® € w,|x} < bP(8 € wlx} 


and decision d, if the reverse inequality holds. The choice of decision is 
immaterial in case of equality. 

In the case of point estimation of a real-valued function g(0) with loss 
function L(0, d) = (g(0) — d)*, the Bayes solution becomes 6(x) = 
E[g(9)|x]. When instead the loss function is L(0, d) = |g(8) — d|, the 
Bayes estimate 8(x) is any median of the conditional distribution of 
g(9) given x. 


The Bayes risk r(p,8) can be written as JUSL, 8(x))m(0]x) 40] x 


p(x) dx, where p(x) = fp(0^) pe (x) d0'. 
(ii): The conditional expectation [L(0, d9)m(8|x) d0 reduces to aP(0€ 
|x}, and similarly for d] 


9. (i) 


(ii) 


As an example in which randomization reduces the maximum risk, 
suppose that a coin is known to be either standard (HT) or to have heads 
on both sides (HH). The nature of the coin is to be decided on the basis 
of a single toss, the loss being 1 for an incorrect decision and 0 for a 
correct one. Let the decision be HT when T is observed, whereas in the 
contrary case the decision is made at random, with probability p for HT 
and 1 — p for HH. Then the maximum risk is minimized for p = 3. 

A genetic setting in which such a problem might arise is that of a couple, 
of which the husband is either dominant homozygous (AA) or hetero- 
zygous (Aa) with respect to a certain characteristic, and the wife is 


.homozygous recessive (aa). Their child is heterozygous, and it is of 


importance to determine to which genetic type the husband belongs. 
However, in such cases an a priori probability is usually available for the 
two possibilities. One is then dealing with a Bayes problem, and randomi- 
zation is no longer required. In fact, if the a priori probability is p that 
the husband is dominant, then the Bayes procedure classifies him as such 
if p > 4 and takes the contrary decision if p < }. 
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Unbiasedness and minimax. Let Q= Q, U Q, where Ro, Qy are mutually 
exclusive, and consider a two-decision problem with loss function L(0, d;) = a, 
for 0 € Q; (j + i) and L(0, d;) = 0 for 0 € Q, (i = 0,1). 


G) Any minimax procedure is unbiased. 

(ii) The converse of (i) holds provided P,(A) is a continuous function of 0 
for all A, and if the sets 2) and Q, have at least one common boundary 
point. 


[@: The condition of unbiasedness in this case is equivalent to sup R;(0) < 
aga,/(ag + a). That this is satisfied by any minimax procedure is seen by 
comparison with the procedure &(x)- d, or = d, with probabilities a,/(4o 
+ a,) and ag/(ao + 41) respectively. 

(ii): If 8) is a common boundary point, continuity of the risk function implies 
that any unbiased procedure satisfies R5(8%) = aga,/(ao + 41) and hence 
supR5(0) = a941/(4o + 41)-] 


Invariance and minimax. Let a problem remain invariant relative to the 
groups G, G, and G* over the spaces F, Q, and D respectively. Then a 
randomized procedure Y, is defined to be invariant if for all x and g the 
conditional distribution of Y, given x is the same as that of g*~'Y,,. 


(i) Consider a decision procedure which remains invariant under a finite 
group G = {g1,---,8y}- Ifa minimax procedure exists, then there exists 
one that is invariant. 


(ii) This conclusion does not necessarily hold for infinite groups, as is shown 
by the following example. Let the parameter space Q consist of all 
elements 6 of the free group with two generators, that is, the totality of 
formal products 7... m, (n = 0,1,2,..-) where each v, is one of the 
elements a, a~!, b, b^! and in which all products aa~', a~'a, bb !, and 
b-!b have been canceled. The empty product (n = 0) is denoted by e. 
The sample point X is obtained by multiplying 6 on the right by one of 
the four elements a, a~}, b, b~} with probability } each, and canceling if 
necessary, that is, if the random factor equals 7, 1 The problem of 
estimating @ with L(0,d) equal to 0 if d = 0 and equal to 1 otherwise 
remains invariant under multiplication of X, 0, and d on the left by an 
arbitrary sequence 7. ,,... 7 57. (m= 0,1,...). The invariant proce- 
dure that minimizes the maximum risk has risk function R(0,8) = i. 
However, there exists a noninvariant procedure with maximum risk i: 


[(i): If Y, is a (possibly randomized) minimax procedure, an invariant minimax 
procedure Y; is defined by P(Y/ = d) = EIC; P(Y,, = g'd)/N. 

(ii): The better procedure consists in estimating 0 to be 7,...7,., When 
m,...T, is observed (k > 1), and estimating @ to be a, a! b,b^! with 
probability 1 each in case the identity is observed. The estimate will be correct 
unless the last element of X was canceled, and hence will be correct with 
probability 2 7.] 
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Section 7 
Let X have probability density p(x) with @ one of the values 0,,..., 9,, 
and consider the problem of determining the correct value of @, so that 
the choice lies between the n decisions d, = 0,,..., d, = 9, with gain 
a(0,) if d; = 6, and 0 otherwise. Then the Bayes solution (which maxi- 
mizes the average gain) when @ is a random variable taking on each of the 
n values with probability 1/n coincides with the maximum-likelihood 
procedure. 
Let X have probability density p(x) with 0 < 0 < 1. Then the maxi- 
mum-likelihood estimate is the mode (maximum value) of the a posteriori 
density of 8 given x when @ is uniformly distributed over (0, 1). 


Let X,,..., X, be a sample from N(£, 6?), and consider the problem of 
deciding between w):§<0 and w,:€>0. If X=Lx/n and C= 
(a; /a9) ^, the likelihood-ratio procedure takes decision dy or d, as 


Ynx 


VE(x,- x) 
where k= — (C—1 it C » 1 and k - (1- C)/C it C « 1. 


For the problem of deciding between «y: 9 < % and w,: e > 0, the 
likelihood ratio procedure takes decision dy or d, as 


«k or >k, 


I(x- X) 


nop 


«iu orsc ok, 


where k is the smaller root of the equation Cx = e*-! if C > 1, and the 
larger root of x = Ce*-! if C < 1, where C is defined as in (i). 


Section 8 


14. Admissibility of unbiased procedures. 


© 
Gi) 


Under the assumptions of Problem 10, if among the unbiased procedures 
there exists one with uniformly minimum tisk, it is admissible. 

That in general an unbiased procedure with uniformly minimum risk need 
not be admissible is seen by the following example. Let X have a Poisson 
distribution truncated at 0, so that Py{ X= x) = 8*e-* /[xY1 — e7*)) 
for x — 1,2,... . For estimating y(0) = e? with loss function L(0, d) 
— (d — e-5)^, there exists a unique unbiased estimate, and it is not 
admissible. 


[(ii): The unique unbiased estimate 8)(x) = (—1)**? is dominated by 8, (x) 
=0or1 as x is even or odd.] 
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15. Admissibility of invariant procedures. If a decision problem remains invariant 
under a finite group, and if there exists a procedure 6) that uniformly 
minimizes the risk among all invariant procedures, then à, is admissible. 
[This follows from the identity R(0,8) = R(Z0, g*5g_*) and the hint given in 
Problem 11(i).] 


16. (i) Let X take on the values 0 — 1 and 0 + 1 with probability } each. The 
problem of estimating @ with loss function L(0, d) = min(|8 — d|, 1) 
remains invariant under the transformation gX = X +c, g0 = 9- c, 
g'd- d- c. Among invariant estimates, those taking on the values 
X —1 and X+1 with probabilities p and q (independent of X) 
uniformly minimize the risk. 
(i) That the conclusion of Problem 15 need not hold when G is infinite 
follows by comparing the best invariant estimates of (i) with the estimate 
8,(x) which is X + 1 when X < 0 and X — 1 when X z 0. 


Section 9 
17. In n independent trials with constant probability p of success, let X, — lor0 
as the ith trial is a success or not: Then Y7.., X, is minimal sufficient. 
[Let T = X X, and suppose that U = f(T) is sufficient and that f(k) = >+- 
= f(k,) = u. Then P(T = t|U = u} depends on p.) 


18. (i) Let X,...,X, be a sample from the uniform distribution U(0, 0), 
0 « 6 < oo, and let T = max(X;,..., X,). Show that T is sufficient, once 
by using the definition of sufficiency and once by using the factorization 
criterion and assuming the existence of statistics Y, satisfying (17)-(19). 

(i) Let X,,..., X, be a sample from the exponential distribution E(a, b) 
with density (1/b)e- 7^ when x > a (—oo « a € oo, 0 « b). Use 
the factorization criterion to prove that (min(X,,..., X,), L7-1X;) is 
sufficient for a,b, assuming the existence of statistics Y, satisfying 


0709). 
19. A statistic T satisfying (17)-(19) is sufficient if and only if it satisfies (20). 
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CHAPTER 2 


The Probability 
Background 


1. PROBABILITY AND MEASURE 


The mathematical framework for statistical decision theory is provided by 
the theory of probability, which in turn has its foundations in the theory of 
measure and integration. The present and following sections serve to define 
some of the basic concepts of these theories, to establish some notation, and 
to state without proof some of the principal results. In the remainder of the 
chapter, certain special topics are treated in more detail. 

Probability theory is concerned with situations which may result in 
different outcomes. The totality of these possible outcomes is represented 
abstractly by the totality of points in a space Z. Since the events to be 
studied are aggregates of such outcomes, they are represented by subsets of 
Z. The union of two sets C,,C, will be denoted by C, U C,, their 
intersection by C, N C,, the complement of C by C — Z'— C, and the 
empty set by 0. The probability P(C) of an event C is a real number 
between 0 and 1; in particular 


(1) P(0)=0 and P(£)-1. 


Probabilities have the property of countable additivity, 


Q  P(Uc)-XP(C) it GnG-0 forall i+j. 


Unfortunately it turns out that the set functions with which we shall be 
concerned usually cannot be defined in a reasonable manner for all subsets 
of Z if they are to satisfy (2). It is, for example, not possible to give a 
reasonable definition of “area” for all subsets of a unit square in the plane. 
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The sets for which the probability function P will be defined are said to 
be “measurable”. The domain of definition of P should include with any set 
C its complement C, and with any countable number of events their union. 
By (1), it should also include 2. A class of sets that contains Z and is 
closed under complementation and countable unions is a o-field. Such a 
class is automatically also closed under countable intersections. 

The starting point of any probabilistic considerations is therefore a space 
2, representing the possible outcomes, and a o-field € of subsets of Z, 
representing the events whose probability is to be defined. Such a couple 
(Z, €) is called a measurable space, and the elements of € constitute the 
measurable sets. A countably additive nonnegative (not necessarily finite) set 
function p defined over € and such that (0) = 0 is called a measure. If it 
assigns the value 1 to Z, it is a probability measure. More generally, p is 
finite if u(2) < oo and o-finite if there exist C}, C;,... in € (which may 
always be taken to be mutually exclusive) such that UC; = Z and u(C;) < oo 
for i=1,2,.... Important special cases are provided by the following 
examples. 


Example 1. Lebesgue measure. Let Z be the n-dimensional Euclidean space 
E,, and € the smallest o-field containing all rectangles* 


R= (C22) a< z S bi FAL). 


The elements of € are called the Borel sets of E,. Over € a unique measure p can 
be defined, which to any rectangle R assigns as its measure the volume of R, 


a(R) = TCh - a). 


The measure p can be completed by adjoining to € all subsets of sets of measure 
zero. The domain of p is thereby enlarged to a o-field €', the class of Lebesgue- 
measurable sets. The term Lebesgue measure is used for p both when it is defined 
over the Borel sets and when it is defined over the Lebesgue-measurable sets. 


This example can be generalized to any nonnegative set function v, which 
is defined and countably additive over the class of rectangles R. There exists 
then, as before, a unique measure p Over (Z, €) that agrees with v for all 
R. This measure can again be completed; however, the resulting o-field 
depends on p and need not agree with the o-field €' obtained above. 


Example 2. Counting measure, Suppose that Z is countable, and let @ be the 
class of all subsets of Z. For any set C, define p(C) as the number of elements of C 


*If a(z) is a statement concerning certain objects z, then {z: 7(z)} denotes the set of all 
those z for which 7(z) is true. 
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if that number is finite, and otherwise as +00. This measure is sometimes called 
counting measure. 


In applications, the probabilities over (Z^, @) refer to random experi- 
ments or observations, the possible outcomes of which are the points 
z € Z. When recording the results of an experiment, one is usually inter- 
ested only in certain of its aspects, typically some counts or measurements. 
These may be represented by a function T taking values in some space 7. 

Such a function generates in 7 the o-field @’ of sets B whose inverse 
image 


C=T"(B) = (z:ze£,T(z) e B) 


is in €, and for any given probability measure P over (2°, €) a probability 
measure Q over (7, 4^) defined by 


(3) Q(B) = P(T(B)). 


Frequently, there is given a o-field @ of sets in .7 such that the 
probability of B should be defined if and only if B € 4. This requires that 
T-XB) € € for all B € 4, and the function (or transformation) T from 
(Z, €) into* (.Z, 4) is then said to be € measurable. Another implication 
is the sometimes convenient restriction of probability statements to the sets 
B € F even though there may exist sets B € 4 for which T (B) € € and 
whose probability therefore could be defined. 

Of particular interest is the case of a single measurement in which the 
function T is real-valued. Let us denote it by X, and let » be the class of 
Borel sets on the real line Z. Such a measurable real-valued X is called a 
random variable, and the probability measure it generates over (£^, s) will 
be denoted by P* and called the probability distribution of X. The value 
s measure assigns to a set A € 57 will be denoted interchangeably by 
P^(A) and P(X € A). Since the intervals (x:x <a} are in o, the 
probabilities F(a) = P(X < a) are defined for all a. The function F, the 
cumulative distribution function (cdf) of X, is nondecreasing and continuous 
on the right, and F(—09)-— 0, F(+00)=1. Conversely, if F is any 
function with these properties, a measure can be defined over the intervals 
by P{a< x < b) = F(b) — F(a). It follows from Example 1 that this 
measure uniquely determines a probability distribution over the Borel sets. 
Thus the probability distribution P* and the cumulative distribution func- 
tion F uniquely determine each other. These remarks extend to probability 


*The term into indicates that the range of T is in J; if T(2^) = 7, the transformation is 
said to be from Z onto J. 
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distributions over an n-dimensional Euclidean space, where the cumulative 
distribution function is defined by 


F(a,,.--,4,) = P{X% x a,,..., X, € a). 


In concrete problems, the space (2°, €), corresponding to the totality of 
possible outcomes, is usually not specified and remains in the background. 
The real starting point is the set X of observations (typically vector-valued) 
that are being recorded and which constitute the data, and the associated 
measurable space (2, s), the sample space. Random variables or vectors 
that are measurable transformations T from (%, s) into some (7, Z) are 
called statistics. The distribution of T is then given by (3) applied to all 
B € Z. With this definition, a statistic is specified by the function T and the 
o-field Z. We shall, however, adopt the convention that when a function T 
takes on its values in a Euclidean space, unless otherwise stated the o-field 
@ of measurable sets will be taken to be the class of Borel sets. It then 
becomes unnecessary to mention it explicitly or to indicate it in the 
notation. 

The distinction between statistics and random variables as defined here is 
slight. The term statistic is used to indicate that the quantity is a function of 
more basic observations; all statistics in a given problem are functions 
defined over the same sample space (F, £). On the other hand, any 
real-valued statistic T is a random variable, since it has a distribution over 
(7, 2), and it will be referred to as a random variable when its origin is 
irrelevant. Which term is used therefore depends on the point of view and to 
some extent is arbitrary. 


2. INTEGRATION 


According to the convention of the preceding section, a real-valued function 
f defined over (X, ) is measurable if f^ (B) € »£ for every Borel set B 
on the real line. Such a function f is said to be simple if it takes on only a 
finite number of values. Let p be a measure defined over (Z, 7), and let f 
be a simple function taking on the distinct values 4;,..., a,, on the sets 
A,..., Aj, which are in a, since f is measurable. If p(4;) < oo when 
a, + 0, the integral of f with respect to p is defined by 


(4) "n = Lapn(A))- 


Given any nonnegative measurable function f, there exists a nondecreas- 
ing sequence of simple functions f, converging to f. Then the integral of f 
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is defined as 


(5) [fu = tim ff, du, 


which can be shown to be independent of the particular sequence of f,'s 
chosen. For any measurable function f its positive and negative parts 


(6 — f*(x) = max[ f(x),0] and f(x) = max[ -/(),0] 


are also measurable, and 
f(x) =f*(x) - f: (x): 


If the integrals of ft and f~ are both finite, then f is said to be integrable, 
and its integral is defined as 


fidu= ff dp- ff an. 


If of the two integrals one is finite and one infinite, then the integral of f is 
defined to be the appropriate infinite value; if both are infinite, the integral 
is not defined. 


Example 3. Let € be the closed interval [a, b], Z be the class of Borel sets or 
of Lebesgue measurable sets in £', and p be Lebesgue measure. Then the integral of 
f with respect to p is written as [f(x) dx, and is called the Lebesgue integral of f. 
This integral generalizes the Riemann integral in that it exists and agrees with the 
Riemann integral of f whenever the latter exists. 


Example 4. Let € be countable and consist of the points x,, x;,... ; let # be 
the class of all subsets of 2°, and let p assign measure b; to the point x;. Then f is 
integrable provided X f(x,) b; converges absolutely, and /fdy is given by this sum. 


Let P* be the probability distribution of a random variable X, and let T 


be a real-valued statistic. If the function T(x) is integrable, its expectation is 
defined by 


(7) E(T) = f T(x) dP*(x). 


It will be seen from Lemma 2 in Section 3 below that the integration can be 
carried out alternatively in t-space with respect to the distribution of T 
defined by (3), so that also 


(8) E(T) = fae. 
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The definition (5) of the integral permits the basic convergence theorems: 


Theorem 1. Let f, be a sequence of measurable functions, and let 
f(x) > f(x) for all x. Then 


[fin ^ [fan 


if either one of the following conditions holds: 

(i) Lebesgue monotone-convergence theorem: the f,'s are nonnegative 
and the sequence is nondecreasing, 

or 

(ii) Lebesgue dominated-convergence theorem: there exists an integrable 
function g such that |f,(x)| < g(x) for all n and x. 


For any set A €, let I, be its indicator function defined by 


(9) I,(x)=1lor0 as xeA or x € A, 
and let 
(10) [fan - ff. an. 


If p is a measure and f a nonnegative measurable function over (2, £), 
then 


(11) »(A) = ffu 


defines a new measure over (Z, ). The fact that (11) holds for all 4 € » 
is expressed by writing 


? dv 
(12) dv=fdp or ini 


Let p and » be two given o-finite measures over (F, x). If there exists a 
function f satisfying (12), it is determined through this relation up to sets of 
measure Zero, since 


[fen - [st forall A4 €. 
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implies that f=g ae. p.* Such an f is called the Radon-Nikodym 
derivative of v with respect to p, and in the particular case that v is a 
probability measure, the probability density of v with respect to p. 

The question of existence of a function f satisfying (12) for given 
measures p and v is answered in terms of the following definition. A 
measure v is absolutely continuous with respect to p if 


p(A)=0 implies »(A) =0. 


Theorem 2. (Radon-Nikodym.) If and v are o-finite measures over 
(£, £), then there exists a measurable function f satisfying (12) if and only if 
v is absolutely continuous with respect to p. 


The direct (or Cartesian) product A X B of two sets A and B is the set of 
all pairs (x, y) with x € 4, y € B. Let (X, 7) and (Y, 4) be two 
measurable spaces, and let x Z be the smallest o-field containing all sets 
A X B with A € »£ and B € 4. If p and v are two o-finite measures over 
(£, £) and (Y, 4) respectively, then there exists a unique measure À = 
p X » over (XX Y, LX B), the product of p and v, such that for any 
A €f, BER, 


(13) (A x B) = n(A)v(B). 


Example 5. Let 4,4/ be Euclidean spaces of m and n dimensions, and let 
«, 4 be the o-fields of Borel sets in these spaces. Then ZX & is an (m + n)- 
dimensional Euclidean space, and » X 4 the class of its Borel sets. 


Example 6. Let Z = (X, Y) bea random variable defined over (4 x Y, £ X B), 
and suppose that the random variables X and Y have distributions P*, P” over 
(37,57) and (X, 4). Then X and Y are said to be independent if the probability 
distribution P? of Z is the product P* x PY. 


In terms of these concepts the reduction of a double integral to a 
repeated one is given by the following theorem. 


Theorem 3. (Fubini.) Let p and v be o-finite measures over (X, x) and 


(Y, B) respectively, and let X = p X v. If f(x, y) is integrable with respect 
to À, then 


(i) for almost all (v) fixed y, the function f(x, y) is integrable with 
respect to p, 


Gi) the function {f(x, y) du(x) is integrable with respect to v, and 


Q9 — fre »ax y) = f| ffx») aco] ao). 


*A statement that holds for all points x except possibly on a set of -measure zero is said to 
hold a.e. p; or to hold (, p) if it is desirable to indicate the o-field over which p is defined. 
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3. STATISTICS AND SUBFIELDS 


According to the definition of Section 1, a statistic is a measurable transfor- 
mation T from the sample space (¥, »7) into a measurable space (7, #). 
Such a transformation induces in the original sample space the subfield* 


(15) ty =T(#) ={T(B): BEB}. 


Since the set T ![T(4)] contains A but is not necessarily equal to A, the 
o-field .% need not coincide with » and hence can be a proper subfield of 
3. On the other hand, suppose for a moment that 7= T(4'), that is, that 
the transformation T is onto rather than into 7. Then 


(16) T[r(8)- B foral BES, 


so that the relationship A, = T^ !(B) establishes a 1:1 correspondence 
between the sets of and Z, which is an isomorphism—that is, which 
preserves the set operations of intersection, union, and complementation. 
For most purposes it is therefore immaterial whether one works in the space 
(4, A) or in (7, 4). These generate two equivalent classes of events, and 
therefore of measurable functions, possible decision procedures, etc. If the 
transformation T is only into 7, the above 1:1 correspondence applies to 
the class Z’ of subsets of 7’ = T(4') which belong to 2, rather than to 4 
itself. However, any set B € 4! is equivalent to B' = B N J” in the sense 
that any measure over (2, ) assigns the same measure to B’ as to B. 
Considered as classes of events, X) and 4 therefore continue continue to 
be equivalent, with the only difference that 2 contains several (equivalent) 
representations of the same event. 

As an example, let 2 be the real line and .@ the class of Borel sets, and 
let T(x) = x?. Let J be either the positive real axis or the whole real axis, 
and let Z be the class of Borel subsets of 7. Then 5, is the class of Borel 
sets that are symmetric with respect to the origin. When considering, for 
example, real-valued measurable functions, one would, when working in 
F space, restrict attention to measurable functions of x?. Instead, one could 
remain in the original space, where the restriction would be to the class of 
even measurable functions of x. The equivalence is clear. Which representa- 
tion is more convenient depends on the situation. 

That the correspondence between the sets 4; = T (B) € % and B € 
@ establishes an analogous correspondence between measurable functions 
defined over (€, )) and (7, 4) is shown by the following lemma. 


*We shall use this term in place of the more cumbersome “sub-o-field”. 
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Lemma 1. Let the statistic T from (%, £) into (7, B) induce the 
subfield xf). Then a real-valued measurable function f is £y" measurable if 
and only if there exists a B-measurable function g such that 


f(x) = g[TG)] 
for all x. 
Proof. Suppose first that such a function g exists. Then the set 
{x: f(x) <r} = T^G gl) « 7) 


is in 54, and f is s; measurable. Conversely, if f is #o-measurable, then 
the sets 


S 
2" 


i 
An [eig eo) s \, i=0, +1,42,..., 


are (for fixed n) disjoint sets in f) whose union is 2, and there exist 
B,, € @ such that A,, = T~\(B,,). Let 


Bs = B,, 0 UB,,. 
j*i 


Since A,, and A,, are mutually exclusive for i + j, the set T (Bin N B;,) is 
empty and so is the set T^ (B, N B*). Hence, for fixed n, the sets Bj, are 
disjoint, and still satisfy A,, = T~\(B*). Defining 
i H 
f(x) mem if cA XOT 2, 


one can write 


f) = g[ITG9l. 


where 
i 
g(n-2(g for eB, i-0tl£2.. 
0 otherwise. 


Since the functions 8, are Z-measurable, the set B on which g,(t) con- 
verges to a finite limit is in 2. Let R = T(%) be the range of T. Then for 
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t€R, 
lim g,[T(x)] = lim f,(x) = f(x) 


for all x € Z, so that R is contained in B. Therefore, the function g 
defined by g(t) = lim g,(t) for t E B and g(t) = 0 otherwise possesses the 
required properties. 

The relationship between integrals of the functions f and g above is 
given by the following lemma. 


Lemma 2. Let T be a measurable transformation from (Z, £) into 
(T, 4), u a o-finite measure over (€, £), and g a real-valued measurable 
function of t. If u* is the measure defined over (Z, 4) by 


(17) p*(B)-p[T (8) forall Be4, 


then for any B € &, 
isi f, TO du(x) = fgl) ale) 


in the sense that if either integral exists, so does the other and the two are 
equal. 


Proof. Without loss of generality let B be the whole space 7. If g is 
the indicator of a set By € Z, the lemma holds, since the left- and 
right-hand sides of (18) reduce respectively to p[T-1(B,)] and p*(Bo), 
which are equal by the definition of p*. It follows that (18) holds succes- 
sively for all simple functions, for all nonnegative measurable functions, and 
hence finally for all integrable functions. 


4. CONDITIONAL EXPECTATION AND PROBABILITY 


If two statistics induce the same subfield .%, they are equivalent in the 
sense of leading to equivalent classes of measurable events. This equivalence 
is particularly relevant to considerations of conditional probability. Thus if 
X is normally distributed with zero mean, the information carried by the 
statistics |X|, X2, e- ^, and so on, is the same. Given that |X] = t, X? = 
12, e-  — e°, it follows that X is +t, and any reasonable definition of 
conditional probability will assign probability 4 to each of these values. The 
general definition of conditional probability to be given below will in fact 
involve essentially only s, and not the range space 7 of T. However, when 
referred to 4, alone the concept loses much of its intuitive meaning, and 
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“the gap between the elementary definition and that of the general case 
becomes unnecessarily wide. For these reasons it is frequently more con- 
venient to work with a particular representation of a statistic, involving a 
definite range space (7, #). 

Let P be a probability measure over (2, #7), T a statistic with range 
space (7, 2), and X, the subfield it induces. Consider a nonnegative 
function f which is integrable (4%, P), that is, measurable and P-inte- 
grable. Then f,f4P is defined for all A € »£ and therefore for all Ay € %9. 
It follows from the Radon-Nikodym theorem (Theorem 2) that there exists 
a function f, which is integrable (g, P) and such that 


(19) f, Jap = [^d forall Ay € x, 


and that f, is unique (54, P). By Lemma 1, fọ depends on x only through 
T(x). In the example of a normally distributed variable X with zero mean, 
and T = X?, the function f, is determined by (19) holding for all sets Ao 
that are symmetric with respect to the origin, so that f(x) = AL f(x) + 
fx). 

The function f, defined through (19) is determined by two properties: 


(i) Its average value over any set Ag with respect to P is the same as that 
of f; 

(ii) It depends on x only through T(x) and hence is constant on the sets 
D, over which T is constant. 


Intuitively, what one attempts to do in order to construct such a function 
is to define f(x) as the conditional P-average of f over the set D,. One 
would thereby replace the single averaging process of integrating f repre- 
sented by the left-hand side with a two-stage averaging process such as an 
iterated integral. Such a construction can actually be carried out when X is 
a discrete variable and in the regular case considered in Chapter 1, Section 
9; falx) is. then just the conditional expectation of f(X) given T(x). In 
general, it is not clear how to define this conditional expectation directly. 
Since it should, however, possess properties (i) and (ii), and since these 
through (19) determine f, uniquely (.o%), P), we shall take f;(x) of (19) as 
the general definition of the conditional expectation E{ f( X)|T(x)]. Equiv- 
alently, if f(x) = g[T(x)] one can write 


E[f(X)tt] = E[fC X)|T = 1] = g(t), 


so that E[f(X)|t] is a measurable function defined up to equivalence 
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(48, PT). In the relationship of integrals given in Lemma 2, if u = P* then 
u* = PT, and it is seen that the function g can be defined directly in terms 
of f through 


Q) f f(x)dP'(x) = fg(r)dP'(i) forall Bed, 
T-M(B) B 


which is equivalent to (19). 
So far, f has been assumed to be nonnegative. In the general case, the 
conditional expectation of f is defined as 


E[f(x)|q = EL GOL] = ELA (X). 


Example 7. Order statistics. Let X,,..., X, be identically and independently 
distributed random variables with a continuous distribution function, and let 


T(3,..., Xn) = (Xm X0) 
where xq, € -:* < x,,, denote the ordered x's. Without loss of generality one can 
restrict attention to the points with xq) < **- < xj, since the probability of two 


coordinates being equal is 0. Then % is the set of all m-tuples with distinct 
coordinates, 7 the set of all ordered n-tuples, and and 4 are the classes of 
Borel subsets of 4 and J. Under T^! the set consisting of the single point 
a = (4,,..., a,) is transformed into the set consisting of the n! points (a;,,..., a; ) 
that are obtained from a by permuting the coordinates in all possible ways. It 
follows that .f) is the class of all sets that are symmetric in the sense that if Ao 
contains a point x = (x,,-.., x,), then it also contains all points (x,,,..., x;,): 
For any integrable function f, let 


f) = i f(x 


where the summation extends over the n! permutations of (x;,..., x,). Then fo is 
5/,- measurable, since it is symmetric in its n arguments. Also 


J fne) APC) oo APC) = f, fG ss) PCR) c PG). 


so that fy satisfies (19). It follows that fo(x) is the conditional expectation of f(X) 
given T(x). 

The conditional expectation of f( X) given the above statistic T(x) can also be 
found without assuming the X's to be identically and independently distributed. 
Suppose that X has a density h(x) with respect to a measure p (such as Lebesgue 
measure), which is symmetric in the variables x,,-.-, X, in the sense that for any 
A €. it assigns to the set (x:(X,,.-.,X,) € 4) the same measure for all 
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permutations (i;,-.-,i,). Let 


Ce CREE 
Nx x) A 


fois «287 An) "t 


here and in the sums below the summation extends over the n! permutations 
of (x,,...,x,). The function fọ is symmetric in its n arguments and hence s- 
measurable. For any symmetric set Ao, the integral 


J fe m Qnis) M(B Fn) 
o 
has the same value for each permutation (Xpress xi) and therefore 


S fol ise Gne) dph An) 
1 
= f fol ir. a) EG) Gp xis.. X.) 


= f Pee Xn) Qn) MACH ++» Xn) 


It follows that fj(x) = E[f(X)|T(x)}. 


Equivalent to the statistic T(x) = [EC TEE Xn), the set of order statistics, is 
U(x) = (Ex,,Ex,...,Ex?). This is an immediate consequence of the fact, to be 
shown below, that if T(x?) = £? and U(x?) = u^, then 


T({#}) =U (00) -5 


where {1°} and (u^) denote the sets consisting of the single point 1° and u? 

respectively, and where S consists of the totality of points x = (x,..., x,) ob- 

tained by permuting the coordinates of x? = (x... , x?) in all possible ways. 
That T~'({1°}) = S is obvious. To see the corresponding fact for U~}, let 


Va) = (Ex. Day Y xxn mm x. 
i 


i<j i<j<k 


so that the components of V(x) are the elementary symmetric functions v, = 
Ex; s Un 7 Xy... x, of the n arguments x;,..., x,. Then 


(ease Sr, x cu xtd pora ETE ry. 


Hence V(x°) = v! = (0j,...,v9) implies that V-!((v?)) = S. That then also 
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U` ((u*)) = S follows from the 1 : 1 correspondence between u and v established 
by the relations (known as Newton's identities),* 


Uy bilko  U3U4-3 — 57 33 5 ) AVA + (-1) ‘ko, = STE SH 


It is easily verified from the above definition that conditional expectation 
possesses most of the usual properties of expectation. It follows of course 
from the nonuniqueness of the definition that these properties can hold only 
(2, PT). We state this formally in the following lemma. 


Lemma 3. If T is a statistic and the functions f, g,... are integrable 
(2, P), then a.e. (4, P") 

G) E[af(X) + bg(X)|t] = aE[f(X)|t] + BEES UT: 

Gi) | ELA (T) f CX) = ACO ELF 

(ii) a € f(x) € b(%, P) impliesa < E|fCX)|t] < b; 

(iv) [fil < g f(x) > SONL, P) implies EL CX) > ELANA. 

A further useful result is obtained by specializing (20) to the case that B 
is the whole space .7. One then has 


Lemma 4. If E|f(X)| < oo, and if g(t) = E[f(X)|t], then 
(21) Ef(X) = Eg(T), 


that is, the expectation can be obtained as the expected value of the conditional 
expectation. 


Since P( X € A} = E[1,(X)], where 14 denotes the indicator of the set 
A, it is natural to define the conditional probability of A given T = t by 


(22) P(Alt) = E[L,(X)|q]- 


In view of (20) the defining equation for P(A|t) can therefore be written as 


(23)  P'(40T-(B)) of ent ed) 


= f P(4I) aP") forall BES. 
B 


It is an immediate consequence of Lemma 3 that subject to the appropriate 


*For a proof of these relations see for example Turnbull (1952), Theory of Equations, 5th 
ed., Oliver and Boyd, Edinburgh, Section 32. 
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null-set* qualifications, P( At) possesses the usual properties of probabili- 
ties, as summarized in the following lemma. 


Lemma 5. If T is a statistic with range space (7,4), and 
A, B, Ay, As... are sets belonging to xf, then a.e. (d, P") 

(i) 0< P(Alt) <1; 

(ü) if the sets Aj, Az,.-. are mutually exclusive, 


P(UAJ/) = YP(AIf); 


Gii) A C B implies P(A|t) < P(Blt). 


According to the definition (22), the conditional probability P( 4|r) must 
be considered for fixed A as a @measurable function of t. This is in 
contrast to the elementary definition in which one takes ¢ as fixed and 
considers P(.A|t) for varying A as a set function over »/. Lemma 5 suggests 
the possibility that the interpretation of P(A|f) for fixed t as a probability 
distribution over 7 may be valid also in the general case. However, the 
equality P(A, U A,|t) = P(A,|t) + P(A,|t), for example, can break down 
on a null set that may vary with A, and A,, and the union of all these null 
sets need no longer have measure zero. 

For an important class of cases, this difficulty can be overcome through 
the nonuniqueness of the functions P(Ajt), which for each fixed A are 
determined only up to sets of measure zero in t. Since all determinations of 
these functions are equivalent, it is enough to find a specific determination 
for each A so that for each fixed t these determinations jointly constitute a 
probability distribution over s. This possibility is illustrated by Example 7, 
in which the conditional probability distribution given T(x) — t can be 
taken to assign probability 1/n! to each of the n! points satisfying T(x) = t. 
Sufficient conditions for the existence of such conditional distributions will 


qu in the next section. For counterexamples see Blackwell and Dubins 
5). 


5. CONDITIONAL PROBABILITY DISTRIBUTIONS! 


We shall now investigate the existence of conditional probability distribu- 
tions under the assumption, satisfied in most statistical applications, that Z 
is a Borel set in a Euclidean space. We shall then say for short that Z is 


nus term is used as an alternative to the more cumbersome “set of measure zero." 
This section may be omitted at first reading. Its principal application is in the proof of 
Lemma 8(ii) in Section 7, which in turn is used only in the proof of Theorem 3 of Chapter 4. 
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Euclidean and assume that, unless otherwise stated, is the class of Borel 
subsets of Z. 


Theorem 4. If Z is Euclidean, there exist determinations of the functions 
P(A|t) such that for each t, P(A|t) is a probability measure over £. 


Proof. By setting equal to 0 the probability of any Borel set in the 
complement of Z', one can extend the given probability measure to the class 
of all Borel sets and can therefore assume without loss of generality that Z 
is the full Euclidean space. For simplicity we shall give the proof only in the 
one-dimensional case. For each real x put F(x, t) = P((—oo, x]|t) for 
some version of this conditional probability function, and let 7j, 7»... 
denote the set of all rational numbers in some order. Then r; < r; implies 
that F(r, t) € F(r, t) for all t except those in a null set N;;, and hence that 
F(x, t) is nondecreasing in x over the rationals for all ¢ outside of the null 
set N' UN; Similarly, it follows from Lemma 3(iv) that for all ¢ not ina 
null set N”, as n tends to infinity lim F(r, + 1/n, t) = F(r; t) for i= 


proving that for each fixed A € » 
P*(A|t) = P(A|t) (2, P7). 


By definition of P* this is true whenever A is one of the sets (— 00, x] with 
x rational. It holds next when A is an interval (a, b] = (— oo, b] — (— oo, a] 
with a,b rational, since P* is a measure and P satisfies Lemma 5(ii). 
Therefore, the desired equation holds for the field F of all sets A which are 
finite unions of intervals (a; b] with rational end points. Finally, the class 
of sets for which the equation holds is a monotone class (see Problem 1) and 
hence contains the smallest o-field containing 7, which is 2. The measure 
P*( A|t) over f was defined above for all ¢ not in N' U N". However, since 
neither the measurability of a function nor the values of its integrals is 
affected by its values on a null set, one can take arbitrary probability 
measures over wf for t in N’ U N" and thereby complete the determination. 

If X is a vector-valued random variable with probability distribution P* 
and T is a statistic defined over (2, ), let P *'' denote any version of the 
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family of conditional distributions P(Ajt) over Z guaranteed by Theorem 
4. The connection with conditional expectation is given by the following 
theorem. 


Theorem 5. If X is a vector-valued random variable and E|f( X)| < oo, 
then 


Q4) EL/QOVW]) = ffG)aP? (x) — (2, P^). 


Proof. Equation (24) holds if f is the indicator of any set A € A. It 
then follows from Lemma 3 that it also holds for any simple function and 
hence for any integrable function. 

The determination of the conditional expectation E[ f( X)|t] given by the 
right-hand side of (24) possesses for each ¢ the usual properties of an 
expectation, (i), (iii), and (iv) of Lemma 3, which previously could be 
asserted only up to sets of measure zero depending on the functions f, g,... 
involved. Under the assumptions of Theorem 4 a similar strengthening is 
possible with respect to (ii) of Lemma 3, which can be shown to hold except 
possibly on a null set N not depending on the function A. It will be 
sufficient for the present purpose to prove this under the additional assump- 
tion that the range space of the statistic T is also Euclidean. For a proof 
without this restriction see for example Billingsley (1979). 


Theorem 6. If T is a statistic with Euclidean domain and range spaces 
(4, £) and (T, 4), there exists a determination P*' of the conditional 
probability distribution and a null set N such that the conditional expectation 
computed by 


E[fQOV] = [f(x) aP?(x) 
satisfies for all t € N 
(25) E[h(T)f(X)t] = A(O) EU(OI(. 
Proof. For the sake of simplicity and without essential loss of generality 
suppose that T is real-valued. Let P*!'( 4) be a probability distribution over 


Z for each ż, the existence of which is guaranteed by Theorem 4. For 
B € &, the indicator function J,(t) is Z-measurable and 


fol) dP'(t) = P'(B' n B) = PX(T-!B' n T-18) 


forall B'e 4. 
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Thus by (20) 


I,(t)=P*"(TB) a.e. PT. 


Let B,, n = 1,2,...,be the intervals of 7 with rational end points. Then 
there exists a P-null set N = UN, such that for t € N 


Is (t) = P''(T- 1B,) 


for all n. For fixed t € N, the two set functions P*""(T~'B) and J,(t) are 
probability distributions over 2, the latter assigning probability 1 or 0 to a 
set as it does or does not contain the point t. Since these distributions agree 
over the rational intervals B,, they agree for all B € 2. In particular, for 
t € N, the set consisting of the single point £ is in 4, and if 


AO = (x: T(x) 71), 
it follows that for all t € N 
(26) pXi( 40) = 1. 
Thus 


JITONA) aevo) = f AITO f) a?) 


= h(t) [ f(x) aP? (x) 


for t € N, as was to be proved. 

It is a consequence of Theorem 6 that for all t & N, E[h(T)|t] = h(t) 
and hence in particular P(T € B|t) = 1 or Oaste Bort€ B. 

The conditional distributions P*"' still differ from those of the elemen- 
tary case considered in Chapter 1, Section 9, in being defined over (F, #) 
rather than over the set A and the o-field Z“ of its Borel subsets. 
However, (26) implies that for t € N 


PX(A) = PX'(A n AM). 


The calculations of conditional probabilities and expectations are therefore 
unchanged if for t € N, P*" is replaced by the distribution P, which is 
defined over (A, xf) and which assigns to any subset of A the same 
probability as P*". 
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Theorem 6 establishes for all t € N the existence of conditional probabil- 
ity distributions P*", which are defined over (A, «/) and which by 
Lemma 4 satisfy 


e) suo - f, [f perarn] aero 


for all integrable functions f. Conversely, consider any family of distribu- 
tions satisfying (27), and the experiment of observing first T, and then, if 
T = t, a random quantity with distribution P *. The result of this two-stage 
procedure is a point distributed over (Z^, Z) with the same distribution as 
the original X. Thus P *" satisfies this “functional” definition of conditional 
probability. 

If (X, £) is a product space (7X 9, B x €), then A“ is the product 
of Y with the set consisting of the single point 7. For t € N, the conditional 
distribution P*'' then induces a distribution over (/, €), which in analogy 
with the elementary case will be denoted by P”. In this case the definition 
can be extended to all of by letting P'' assign probability 1 to a 
common specified point yo, for all ż E N. With this definition, (27) becomes 


(28) Bf(T, Y) = I] [fe 5 a") dP™(t). 


As an application, we shall prove the following lemma, which will be 
used in Section 7. 


Lemma 6. Let (7, @) and (V, €) be Euclidean spaces, and let PZ’ be 


a distribution over the product space (X, xf) =(7X Y, BX €). Suppose 
that another distribution P, over (X, s£) is such that 


aP,(t, y) = a(y)b(t) aPs(t, y), 


with a( y) > 0 for all y. Then under P, the marginal distribution of T and a 
version of the conditional distribution of Y given t are given by 


arro) = (O)| fato) anro) ang 
and 


dP?"(y) = a(y) aPo(y) 
fal) ary) 
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Proof. The first statement of the lemma follows from the equation 


P,{T € B}=E,[1,(T)] = Eos (T)a(Y)b(T)] 


- [e(o fats) ani» ] anro. 


To check the second statement, one need only show that for any integrable f 
the expectation E, f(Y, T ) satisfies (28), which is immediate. The denomina- 
tor of dP}" is positive, since a(y) > 0 for all y. 


6. CHARACTERIZATION OF SUFFICIENCY 


We can now generalize the definition of sufficiency given in Chapter 1, 
Section 9. If P = ( P, 0 € Q} is any family of distributions defined over a 
common sample space (%, s), a statistic T is sufficient for P (or for 0) if 
for each A in a there exists a determination of the conditional probability 
function P,(A|t) that is independent of &. As an example suppose that 
X,,..., X, are identically and independently distributed with continuous 
distribution function Fy, 0 € Q. Then it follows from Example 7 that the set 
of order statistics T( X) = (Xo... X(ny) is sufficient for 0. 


Theorem 7. Jf 4 is Euclidean, and if the statistic T is sufficient for 2", 
then there exist determinations of the conditional probability distributions 
P,( A|t) which are independent of 0 and such that for each fixed t, P(A|t) is a 
probability measure over £. 


Proof. This is seen from the proof of Theorem 4. By the definition of 
sufficiency one can, for each rational number r, take the functions F(r, t) to 
be independent of 6, and the resulting conditional distributions will then 
also not depend on 6. 

In Chapter 1 the definition of sufficiency was justified by showing that in 
a certain sense a sufficient statistic contains all the available information. In 
view of Theorem 7 the same justification applies quite generally when the 
sample space is Euclidean. With the help of a random mechanism one can 
then construct from a sufficient statistic T a random vector X’ having the 
same distribution as the original sample vector X. Another generalization of 
the earlier result, not involving the restriction to a Euclidean sample space, 
is given in Problem 12. : 

The factorization criterion of sufficiency, derived in Chapter 1, can be 
extended to any dominated family of distributions, that is, any family 
P = (P, 0 € Q} possessing probability densities py with respect to some 
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o-finite measure p over (2, 2). The proof of this statement is based on the 
existence of a probability distribution À = Xc,P;, (Theorem 2 of the Ap- 
pendix), which is equivalent to P in the sense that for any A € » 


(29) A(4)-0 if and only if P,(A) =0 forall 0 EQ. 

Theorem 8. Let P= { Py, 0 € Q} be a dominated family of probability 
distributions over (X, £), and let X = Le;Pp, satisfy (29). Then a statistic T 
with range space (T, B) is sufficient for 2 if and only if there exist 
nonnegative B-measurable functions gy(t) such that 
(30) dP,(x) = go[T(x)] 4A(x) 
for all 0 E€ Q. 


Proof. Let s£, be the subfield induced by T, and suppose that T is 
sufficient for 0. Then for all 0 € Q, Ay € Lo, and A € X 


J PAlrG9) d) = Pol 4 0 4): 
and since A = Xc;P,, 
J PAIT) dd(x) =A(4 N Ao), 


so that P( A|T(x)) serves as conditional probability function also for A. Lut 
go(T(x)) be the Radon-Nikodym derivative dP,(x)/d A(x) for (Lo, A). To 
prove (30) it is necessary to show that gy(T(x)) is also the derivative of P; 
for (sf, A). If Ag is put equal to Z in the first displayed equation, this 
follows from the relation 


P,(A) = [P(A|T(x)) dPo(x) = fE [ra GOImG9] ab G) 
= JELITO] eG) a(x) 
= f Erle (TOLET) A(x) 


= [eo(T(x)) L(x) 446) = f go(T(x)) G2. 
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Here the second equality uses the fact, established at the beginning of the 
proof, that P(A|T(x)) is also the conditional probability for À; the third 
equality holds because the function being integrated is ,-measurable and 
because dP, = gg dÀ for (Xo, À); the fourth is an application of Lemma 
3(ii); and the fifth employs the defining property of conditional expectation. 

Suppose conversely that (30) holds. We shall then prove that the condi- 
tional probability function P4(A4|f) serves as a conditional probability 
function for all P € 2. Let go(T(x)) = dP4(x)/dÀ(x) on s and for fixed 
A and 6 define a measure v over & by the equation dv = I4 dP. Then over 
ty, dv(x)/dP9(x) = E,{1,(X)|T(x)], and therefore 


dv(x) 


dX(x) = P,[A|T(x)] go(T(x)) ^ over s. 


On the other hand, dv(x)/dA(x) = I4(x)go(T(x)) over £, and hence 


dv(x) 


adA(x) E, [n GO gs (CrOO)|TG)] 


= P,[A|T(x)] go(T(x)) ^ over s. 


It follows that PC A[T(x)) ge(T(x)) = Po A|T(x))So(T(x)) (Xo, ^) and 
hence (.3£,, Pp). Since gy(T(x)) # 0 (Xo, P,), this shows that P,CA[T(x)) 
= P,(A|T(x)) (Xo, Py), and hence that P,(A|T(x)) is a determination of 
P,( A|T(x)). 

Instead of the above formulation, which explicitly involves the distribu- 
tion A, it is sometimes more convenient to state the result with respect to a 
given dominating measure p. 


Corollary 1. (Factorization theorem.) If the distributions Py of P have 
probability densities p, = dP,/dp. with respect to a o-finite measure p, then T 
is sufficient for P if and only if there exist nonnegative B-measurable functions 
go on T and a nonnegative s£measurable function h on € such that 
(31) px) = golT(x)] h(x) (a) 

Proof. Let À = Ec;P,, satisfy (29). Then if T is sufficient, (31) follows 
from (30) with h = dd/dp. Conversely, if (31) holds, 

dd(x) = Ec.go [TG)] A(x) dn(») = k[T(x)] h(x) dn(x) 


and therefore dP,(x) = gf(T(x)) dA(x), where gf(t) = go(t)/k(t) when 
k(t) > 0 and may be defined arbitrarily when k(t) = 0. 
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For extensions of the factorization theorem to undominated families, see 
Ghosh, Morimoto, and Yamada (1981) and the literature cited there. 


7. EXPONENTIAL FAMILIES 


An important family of distributions which admits a reduction by means of 
sufficient statistics is the exponential family, defined by probability densities 
of the form 


k 
(32) P(x) = ctn r oor ae 
j- 


with respect to a o-finite measure p over a Euclidean sample space (X, <). 
Particular cases are the distributions of a sample X = (Xj,..-, X,) from a 
binomial, Poisson, or normal distribution. In the binomial case, for exam- 
ple, the density (with respect to counting measure) is 


()ra-5"*-a - prep l|: : Jo. 


Example 8 If Y,,..., Y, are independently distributed, each with density (with 
respect to Lebesgue measure) 


y Mexp[ -y/(26?)] 
CPT ' 


then the joint distribution of the Y's constitutes an exponential family. For c = 1, 
(33) is the density of the x?-distribution with f degrees of freedom; in particular, 
for f an integer this is the density of Z4, X7, where the X's are a sample from the 
normal distribution N(0, 1). 


(33) py) = > 0, 


Example 9. Consider n independent trials, each of them resulting in one of the 
s outcomes E... E, with probabilities p,,..., p, respectively. If X, is 1 when 
the outcome of the ith trial is E, and 0 otherwise, the joint distribution of the X’s is 


PENX = xus- Xps = x,,) = PEPE... pe, 
where all x;;=0 or 1 and £,x,,=1. This forms an exponential family with 
T(x) = Efax; (J =1,..58 = 1). The joint distribution of the T's is Ces 
nomial distribution M(n; p,,..., p,) given by 
(34) P(T, «tc mop 


Nate [i0 0 e stel Rd aiat 
al... ant - e t)! 


Xp? ...pi(U- E ott 


2.7] EXPONENTIAL FAMILIES 57 


If X,,..., X, is a sample from a distribution with density (32), the joint 
distribution of the X’s constitutes an exponential family with the sufficient 
statistics 15 ,7,(Xj), j = 1,..., k. Thus there exists a k-dimensional suffi- 
cient statistic for (X,,..., X,) regardless of the sample size. Suppose 
conversely that X,,..., X, is a sample from a distribution with some 
density p(x) and that the set over which this density is positive is 
independent of 0. Then under regularity assumptions which make the 
concept of dimensionality meaningful, if there exists a k-dimensional suffi- 
cient statistic with k < n, the densities p(x) constitute an exponential 
family. For a proof and discussion of regularity conditions see, for example, 
Barankin and Maitra (1963), Brown (1964), Barndorff-Nielsen and Pedersen 
(1968), and Hipp (1974). 

Employing a more natural parametrization and absorbing the factor h(x) 
into p, we shall write an exponential family in the form dP,(x) = 
Po(x) du(x) with 


k 
(35) p(x) = cto] Eun 


For suitable choice of the constant C(@), the right-hand side of (35) is a 
probability density provided its integral is finite. The set € of parameter 
points 0 = (0,,...,9,) for which this is the case is the natural parameter 
space of the exponential family (35). 

Optimum tests of certain hypotheses concerning any 6, are obtained in 
Chapter 4. We shall now consider some properties of exponential families 
required for this purpose. 


Lemma 7. The natural parameter space of an exponential family is 
convex. 


Proof. Let (6,,...,0,) and (0,,...,0,) be two parameter points for 
which the integral of (35) is finite. Then by Hólder's inequality, 


fool E [a6 + à - 95]769] del) 


i [feel Een aoo] [fest zem ool aic] - < 00 


for any 0 « a « 1. 

If the convex set @ lies in a linear space of dimension < k, then (35) can 
be rewritten in a form involving fewer than k components of T. We shall 
therefore, without loss of generality, assume Q to be k-dimensional. 
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It follows from the factorization theorem that T(x) = (T;(x)..... T,(x)) 
is sufficient for P = (P5, 6 € €). 


Lemma 8. Let X be distributed according to the exponential family 
» Lj 
abi (x) = C0, ves Fouls) +E sae) "n 
i=1 j=l 


Then there exist measures Xy and v, over s- and r-dimensional Euclidean 
space respectively such that 


G) the distribution of T =(T,,...,T,) is an exponential family of the 
form 


(36) dPI (t) = C(8, e| È on) EH 


(i) the conditional distribution of U = (U,,...,U,) given T = t is an 
exponential family of the form 


(37) db "(u) = e| DOM "ht 


and hence in particular is independent of 9. 


Proof. Let (09, 9?) be a point of the natural parameter space, and let 
p* = PS yv. Then 


c(0, 9) 


Pi (x) = TO, 0°) 


xen É (6,- u(x) + X (2, - nico |arco. 
j=1 


i= 


and the result follows from Lemma 6, with 


dd4(t) = exp( - Er) | f b (6, - a) anite] dP% e(t) 


and 


dv(u) = exp( - Eou) dPjil'y (u). 
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Theorem 9. Let $ be any function on (X, £) for which the integral 
k 
(38) folop] È 6756) | du(x) 
j=l 


considered as a function of the complex variables 0, = £; + in, (j = 1,.-.,k) 
exists for all (£i, ..., £y) € Q and is finite. Then 

(i) the integral is an analytic function of each of the 0’s in the region R of 
parameter points for which (&y,...,£,) is an interior point of the natural 
parameter space 2; 

(ii) the derivatives of all orders with respect to the 0’s of the integral (38) 
can be computed under the integral sign. 


Proof. Let (£0,...,£Q) be any fixed point in the interior of Q, and 
consider one of the variables in question, say 6;. Breaking up the factor 


e Gexp[(£2 + ind) T (x) + -+ + (ER imp) G9] 
into its real and complex part and each of these into its positive and 
negative part, and absorbing this factor in each of the four terms thus 


obtained into the measure p, one sees that as a function of 0, the integral 
(38) can be written as 


[exei 60] d(x) - fexelé 69] dia) 


+i fexp[&TG)] dus(x) — i fexp[4.7,(x)] du,(x). 
It is therefore sufficient to prove the result for integrals of the form 
«(6 = fex G2] n2. 
Since (£9,..., £0) is in the interior of Q, there exists à > 0 such that (4,) 


exists and is finite for all 8, with |, — é?| < 6. Consider the difference 
quotient 


¥(0,) — ¥ (6?) exp[4,7;()] a exp[0°7,(x)] i 
puer E E en doaa aa id; 
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The integrand can be written as 


exp|(0, — 6?)T,(x)| - 1 
exp[6?,()] [si cnn t : 


Applying to the second factor the inequality 


= 8 
exp(az) |, ee) E Eea; 


Zz 


the integrand is seen to be bounded above in absolute value by 
1 1 
sevo? + 8|T;|)|< gs lel? + 8)1] + ex[(6? - 6)7,]| 


for |ð, — 0| < 8. Since the right-hand side is integrable, it follows from the 
Lebesgue dominated-convergence theorem [Theorem 1(ii) that for any 
sequence of points 0(? tending to 0? the difference quotient of y tends to 


[nGoero[o?r G9] an). 


This completes the proof of (i), and proves (ii) for the first derivative. The 
proof for the higher derivatives is by induction and is completely analogous. 


8. PROBLEMS 
Section 1 


1. Monotone class. A class F of subsets of a space is a field if it contains the 
whole space and is closed under complementation and under finite unions; a 
class is monotone if the union and intersection of every increasing and 
decreasing sequence of sets of Æ is again in M. The smallest monotone class 
Ay Pippi a given field F coincides with the smallest o-field « contain- 
ing F. 

[One proves first that JA is a field. To show, for example, that 4 N B € Mo 
when A and B are in JA, consider, for a fixed set A € F, the class JA , of all 
B in Mg for which A B € A. Then M, is a monotone class containing 
F, and hence #, = My. Thus AN BEM, for all B. The argument can 
now be repeated with a fixed set B € J and the class M, of sets A in Æo 
for which A N B € Mg. Since M, is a field and monotone, it is a o-field 
containing 9 and hence contains 3. But any o-field is a monotone class so 
that also Mo is contained in ».] 


2.8] 


2. 
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Section 2 
Radon-Nikodym derivatives. 


(i) If A and p are o-finite measures over (%,.o/) and p is absolutely 
continuous with respect to A, then 


[ran = fr d^ 


for any p-integrable function f. 


Gi) If A, p, and v are o-finite measures over (4,7) such that v is 
absolutely continuous with respect to » and p with respect to A, then 


dv dv dy 
See a.e. À. 

dX dy dÀ 

(ii) If p and v are o-finite measures, which are equivalent in the sense that 


each is absolutely continuous with respect to the other, then 


(v) If pp, k — 1,2,..., and p are finite measures over (X, 2) such that 
Y? jp, (A) = (A) for all A € s, and if the p, are absolutely continu- 
ous with respect to a o-finite measure À, then p is absolutely continuous 
with respect to A, and 


n n 

d d} h 
Em i, E" a AE 
dx kc, dA! neo dÀ dÀ Vd 


[G): The equation in question holds when f is the indicator of a set, hence 
when f is simple, and therefore for all integrable f. 
(ii): Apply (i) with f = d»/dp.] 
If f(x) > 0 for all x € S and p is o-finite, then fsfdp = 0 implies p(S) = 0. 
[Let S, be the subset of S on which f(x) > 1/n. Then p(S) < Lp(S,) and 
(S,) < nfs, fd < nfsfdp = 0] 

Section 3 
Let (X, of) be a measurable space, and a a o-field contained in X. Suppose 
that for any function T, the o-field 2 is taken as the totality of sets B such 
that T7! (B) € s. Then it is not necessarily true that there exists a function T 
such that T^! (4) — %. 
[An example is furnished by any .% such that for all x the set consisting of 
the single point x is in %,.] 


62 THE PROBABILITY BACKGROUND [2.8 
Section 4 
5. (i) Let 2 be any family of distributions X = (X;,..., X,) such that 


P(X, Xi Xx Xo Xx) €4) = P{(X,,..., X,) € 4) 


for all Borel sets A and all i = 1,..., n. For any sample point (x,,..., x,) 
define (yj,..., Yq) = Gp Xi Xn Xp X21), Where x; = xa) = 
min(x;,..., x,). Then the conditional expectation of f(X) given Y = y 
is 


1 
fo») =O) * f») 


s EO x9] 


(ii) Let G= (g;,..., g,) be any group of permutations of the coordinates 
X... X, Of a point x in n-space, and denote by gx the point obtained 
by applying g to the coordinates of x. Let 4 be any family of 
distributions P of X = (X,,..., X,) such that 


(39) P{gX€A}=P{XEA} forall geG. 


For any point x let t = T(x) be any rule that selects a unique point 
from the r points g,x, k — 1,...,r (for example the smallest first 
coordinate if this defines it uniquely, otherwise also the smallest second 
coordinate, etc.). Then 


EOI] =} E fuo). 


(iii) Suppose that in (ii) the distributions P do not satisfy the invariance 
condition (39) but are given by 
dP(x) =h(x) du(x), 


where p is invariant in the sense that p (x: gx € A} = p(A). Then 


X fCGget) AC get) 
E[f(x)|d - M —— 
EGO 


Section 5 


6. Prove Theorem 4 for the case of an n-dimensional sample space. 
[The condition that the cumulative distribution function is nondecreasing is 
replaced by P{x, < X, < x,..., x, < X, < x4} > 0; the condition that it is 
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continuous on the right can be stated as lim,, , ,, F(x; + 1/m,..., x, + 1/m) 
mE (Xp os Xe) 


7. Let € — (x Z, and suppose that Py, P, are two probability distributions 
given by 


dPy(y,t) 7 f(y) (t) du») dv(1), 
aP,(y,t) =h(y, t) du(y) dv(t), 


where A(y, t)/f(y)g(t) < oo. Then under P, the probability density of Y 
with respect to p is 


TC h.T) |) _ 
n) fosse | 


[We have 


Y hy, t) 
RO) = f, wD) =O, 55400 904 


Section 6 
8. Symmetric distributions. 


(i) Let P be any family of distributions of X = (X,,..., Xn) which are 
symmetric in the sense that 


PASER € 4) = P((X,..... X.) € 4) 


for all Borel sets A and all permutations (i,,...,i,) of (1,..., n). Then 
the statistic T of Example 7 is sufficient for 2^, and the formula given in 
the first part of the example for the conditional expectation E[ f( X)|T(x)] 
is valid. 

Gi) The statistic Y of Problem 5 is sufficient. 

(ii) Let X,,..., X, be identically and independently distributed according to 
a continuous distribution P € 2, and suppose that the distributions of 
P are symmetric with respect to the origin. Let V, = |X;| and W, = Kj. 
Then (W,,..., W,) is sufficient for 2. 


9. Sufficiency of likelihood ratios. Let Py, P, be two distributions with densities 


Po, Py. Then T(x) = pi (x)/po(x) is sufficient for P = { Fo, P,). 
[This follows from the factorization criterion by writing pı = T: Pos po 7 


1- po.] 
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11. 


12. 
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Pairwise sufficiency. A statistic T is pairwise sufficient for P if it is sufficient 
for every pair of distributions in 2. 


(i) If 2 is countable and T is pairwise sufficient for 2, then T is sufficient 
for 2. 

(i) If £ is a dominated family and T is pairwise sufficient for ?, then T is 
sufficient for 2. 


[(): Let P= { Py, P,,...), and let »/, be the sufficient subfield induced by T. 
Let A = EcP, (c, > 0) be equivalent to 2. For each j = 1,2,... the probabil- 
ity measure N that is proportional to (co/n)P, + cjP, is ; equivalent to 
{P Pi}. Thus "by pairwise sufficiency, the derivative f, — AP co/n) dP) + 

c, 4P,)| is -measurable. Let S; = (x: f(x) = 0) and S = U/.,S,. Then 
Sex, Py(S) = 0, and on g- S the derivative dP,/4Y7 .,c;P, equals 
QUIT fy which is %-measurable. It then follows from Problem 2 that 


is also .4/;-measurable. 
(ii): Let À = 37,0, Py, be equivalent to 2. Then pairwise sufficiency of T 


implies for any % that dP;,/(dPs, + dd) and hence dP, /dd is a measurable 
function of T.] 


If a statistic T is sufficient for P, then for every function f which is 
(#, P,)-integrable for all 0 € Q there exists a determination of the conditional 
expectation function Ej[ f( X)|t] that is independent of 0. 

[If 2 is Euclidean, this follows from Theorems 5 and 7. In general, if f is 
nonnegative there exists a nondecreasing sequence of simple nonnegative 
functions f, tending to f. Since the conditional expectation of a simple 


function can be taken to be independent of Ó by Lemma 3(i), the desired result 
follows from Lemma 3(iv).] 


For a decision problem with a finite number of decisions, the class of 
procedures depending on a sufficient statistic T only is essentially complete. 
[For Euclidean sample spaces this follows from Theorem 4 without any 
restriction on the decision space. For the present case, let a decision procedure 
be given by 8(x) = (8 (x),..., 8? (x)) where 8?(x) is the probability 
with which decision d; is taken ‘when x is observed. If T is sufficient and 
f(t) = ELSO ( X)|t], ‘the procedures 8 and 7 have identical risk functions.] 
[More general versions of this result are discussed, for example, by Elfving 
(1952), Bahadur (1955), Burkholder (1961), LeCam (1964), and Roy and 
Ramamoorthi (1979).] 


2.8] 


13. 


14. 
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Section 7 
Let X, (i — 1,...,5s) be independently distributed with Poisson distribution 
P(À,), and let T; = XX, T; = X,, A = EA. Then T; has the Poisson distribu- 
tion P(A), and the conditional distribution of 7;,..., T, , given Ty = tọ is the 
multinomial distribution (34) with n = tọ and p, = 1;/A. 
[Direct computation.] 


Life testing. Let X,,...,X, be independently distributed with exponential 
density (20)-le-*/7* for x 2 0, and let the ordered X's be denoted by 
Y, < Y, < -- «Y, It is assumed that Y, becomes available first, then Yz, 
and so on, and that observation is continued until Y, has been observed. This 
might arise, for example, in life testing where each X measures the length of 
life of, say, an electron tube, and n tubes are being tested simultaneously. 
Another application is to the disintegration of radioactive material, where n is 
the number of atoms, and observation is continued until r a-particles have 
been emitted. 


(i The joint distribution of Y,,..., Y, is an exponential family with density 


x5. E+ (n-ro 
: i=l 
= 5 0 EE À 
Q8)' (n- i? 20 A cs 
Gi) The distribution of [Z7.;Y, + (n — r)Y]/0 is x? with 2r degrees of 
freedom. 


(iii) Let Y, Yj,... denote the time required until the first, second, .. . event 
occurs in a Poisson process with parameter 1/26’ (see Chapter 1, 
Problem 1) Then Z; = X/8, Z = (Y, - Y)/0, Z, (Y, — 
Y,)/0’,... are independently distributed as x? with 2 degrees of free- 
dom, and the joint density of Y;,..., Y, is an exponential family with 
density 


1 
yv) ORS tes aere 
The distribution of Y,/0' is again x? with 2r degrees of freedom. 

(iv) The same model arises in the application to life testing if the number n 
of tubes is held constant by replacing each burned-out tube with a new 
one, and if Y, denotes the time at which the first tube burns out, Y, the 
time at which the second tube burns out, and so on, measured from some 
fixed time. 


[Gi): The random variables Z; = (n — i + JY, — ¥j-1)/8 (i= 1,...,r) are 
independently distributed as x? with 2 degrees of freedom, and [Ej_1¥; + 
(n = r)¥,1/0 = X112] 
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15. For any 0 which is an interior point of the natural parameter space, the 
expectations and covariances of the statistics 7; in the exponential family (35) 


are given by 
"p meis Lau) (indy. k), 
" 0 
E[r(X) 70] - [£500 En(X)] = I (i, j=1,...,k). 


16. Let Q be the natural parameter space of the exponential family (35), and for 
any fixed £,,4,..., f, (r < k) let Q5, be the natural parameter space of 
the family of conditional distributions given T; ,, = t,,,,..., T, = t. 


(i) Then Q5. 5 contains the projection Qj 


(ii) An example in which 9, 
of densities 


0, of Q onto 6,,..., 9, 


p 


o, is the family 


e, is a proper subset of Qf, 


Pao (X; Y) = C(0,,6,)exp(0ix + y—xy), x, y>0. 
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CHAPTER 3 
Uniformly Most Powerful Tests 


1. STATING THE PROBLEM 


We now begin the study of the statistical problem that forms the principal 
subject of this book,* the problem of hypothesis testing. As the term 
suggests, one wishes to decide whether or not some hypothesis that has been 
formulated is correct. The choice here lies between only two decisions: 
accepting or rejecting the hypothesis. A decision procedure for such a 
problem is called a test of the hypothesis in question. 

The decision is to be based on the value of a certain random variable X, 
the distribution P, of which is known to belong to a class P= { Py, 
9 € €). We shall assume that if @ were known, one would also know 
whether or not the hypothesis is true. The distributions of # can then be 
classified into those for which the hypothesis is true and those for which it is 
false. The resulting two mutually exclusive classes are denoted by H and K, 
and the corresponding subsets of Q by Qy and Qy respectively, so that 
H U K — and Qy U Qk = Q. Mathematically, the hypothesis is equiv- 
alent to the statement that P, is an element of H. It is therefore convenient 
to identify the hypothesis with this statement and to use the letter H also to 
denote the hypothesis. Analogously we call the distributions in K the 
alternatives to H, so that K is the class of alternatives. 

Let the decisions of accepting or rejecting H be denoted by d, and d, 
respectively. A nonrandomized test procedure assigns to each possible value 
x of X one of these two decisions and thereby divides the sample space into 
two complementary regions Sọ and S,. If X falls into So the hypothesis is 
accepted; otherwise it is rejected. The set S, is called the region of 
acceptance, and the set S, the region of rejection or critical region. 


*The related subject of confidence intervals is treated in Chapter 3, Section 5; Chapter 5, 
Sections 6, 7; Chapter 6, Sections 11-13; Chapter 7, Section 8; Chapter 8, Section 6; and 
Chapter 10, Section 4. 
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When performing a test one may arrive at the correct decision, or one 
may commit one of two errors: rejecting the hypothesis when it is true (error 
of the first kind) or accepting it when it is false (error of the second kind). 
The consequences of these are often quite different. For example, if one tests 
for the presence of some disease, incorrectly deciding on the necessity of 
treatment may cause the patient discomfort and financial loss. On the other 
hand, failure to diagnose the presence of the ailment may lead to the 
patient’s death. 

It is desirable to carry out the test in a manner which keeps the 
probabilities of the two types of error to a minimum. Unfortunately, when 
the number of observations is given, both probabilities cannot be controlled 
simultaneously. It is customary therefore to assign a bound to the probabil- 
ity of incorrectly rejecting H when it is true, and to attempt to minimize the 
other probability subject to this condition. Thus one selects a number a 
between 0 and 1, called the /evel of significance, and imposes the condition 
that 


G) PB {6(X)=d,}=P{XES,} <a forall 0 €Q. 


Subject to this condition, it is desired to minimize P,{5(X) = do} for 0 in 
Qx or, equivalently, to maximize 


(2) P,{6(X) =d,} =P{XES,} forall 9EM,. 


Although usually (2) implies that 
(3) sup Py{ X € $,) = a, 
Qn 


it is convenient to introduce a term for the left-hand side of (3): it is called 
the size of the test or critical region S;. The condition (1) therefore restricts 
consideration to tests whose size does not exceed the given level of signifi- 
cance. The probability of rejection (2) evaluated for a given @ in Qy is 
called the power of the test against the alternative 0. Considered as a 
function of @ for all 0 € Q, the probability (2) is called the power function 
of the test and is denoted by B(@). 

The choice of a level of significance a will usually be somewhat arbitrary, 
Since in most situations there is no precise limit to the probability of an 
error of the first kind that can be tolerated. Standard values, such as .01 or 
-05, were originally chosen to effect a reduction in the tables needed for 
carrying out various tests. By habit, and because of the convenience of 
standardization in providing a common frame of reference, these values 
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gradually became entrenched as the conventional levels to use. This is 
unfortunate, since the choice of significance level should also take into 
consideration the power that the test will achieve against the alternatives of 
interest. There is little point in carrying out an experiment which has only a 
small chance of detecting the effect being sought when it exists. Surveys by 
Cohen (1962) and Freiman et al. (1978) suggest that this is in fact the case 
for many studies. Ideally, the sample size should then be increased to permit 
adequate values for both significance level and power. If that is not feasible, 
one may wish to use higher values of a than the customary ones. The 
opposite possibility, that one would like to decrease a, arises when the latter 
is so close to 1 that a can be lowered appreciably without a significant loss 
of power (cf. Problem 50). Rules for choosing a in relation to the attainable 
power are discussed by Lehmann (1958), Arrow (1960), and Sanathanan 
(1974), and from a Bayesian point of view by Savage (1962, pp. 64-66). See 
also Rosenthal and Rubin (1985). 

Another consideration that may enter into the specification of a signifi- 
cance level is the attitude toward the hypothesis before the experiment is 
performed. If one firmly believes the hypothesis to be true, extremely 
convincing evidence will be required before one is willing to give up this 
belief, and the significance level will accordingly be set very low. (A low 
significance level results in the hypothesis being rejected only for a set of 
values of the observations whose total probability under the hypothesis is 
small, so that such values would be most unlikely to occur if H were true.) 

In applications, there is usually available a nested family of rejection 
Tegions, corresponding to different significance levels. It is then good 
practice to determine not only whether the hypothesis is accepted or 
rejected at the given significance level, but also to determine the smallest 
significance level & = á(x), the significance probability or p-value,* at which 
the hypothesis would be rejected for the given observation. This number 
gives an idea of how strongly the data contradict the hypothesis, and 
enables others to reach a verdict based on the significance level of their 
choice (cf. Problem 9 and Chapter 4, Problem 2). For various questions of 
interpretation and some extensions of the concept, see Dempster and 
Schatzoff (1965), Stone (1969), Gibbons and Pratt (1975), Cox (1977), Pratt 
and Gibbons (1981, Chapter 1) and Thompson (1985). The large-sample 
behavior of p-values is discussed in Lambert and Hall (1982), and their 


sensitivity to changes in the model in Lambert (1982). A graphical proce- 
dure for assessing the p-values of simultaneous tests of several hypotheses is 
proposed by Schweder and Spjetvoll (1982). 


*For a related concept, which comp. 


ares the “acceptability” of 
values, see Spjatvoll (1983), ER OUO, Bio ROUGE parameter 
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Significance probabilities, with the additional information they provide, 
are typically more appropriate than fixed levels in scientific problems, 
whereas a fixed predetermined « is unavoidable when acceptance or rejec- 
tion of H implies an imminent concrete decision. A review of some of the 
issues arising in this context, with references to the literature, is given in 
Kruskal (1978). 

A decision making aspect is often imposed on problems of scientific 
inference by the tendency of journals to publish papers only if the reported 
results are significant at a conventional level such as 5%. The unfortunate 
consequences of such a policy have been explored, among others, by 
Sterling (1959) and Greenwald (1975). 

Let us next consider the structure of a randomized test. For any value x 
such a test chooses between the two decisions, rejection or acceptance, with 
certain probabilities that depend on x and will be denoted by $(x) and 
1 — $(x) respectively. If the value of X is x, a random experiment is 
performed with two possible outcomes R and R, the probabilities of which 
are $(x) and 1 — $(x). If in this experiment R occurs, the hypothesis is 
rejected, otherwise it is accepted. A randomized test is therefore completely 
characterized by a function ¢, the critical function, with 0 < (x) < 1 for 
all x. If $ takes on only the values 1 and 0, one is back in the case of a 
nonrandomized test. The set of points x for which (x) = 1 is then just the 
region of rejection, so that in a nonrandomized test $ is simply the indicator 
function of the critical region. 

If the distribution of X is P}, and the critical function $ is used, the 
probability of rejection is 


Eje (X) = f(x) dP4(x), 


the conditional probability (x) of rejection given x, integrated with 
respect to the probability distribution of X. The problem is to select $ so as 
to maximize the power 


(4) B,(8) = Ejo(X) forall 0E Qy 
Subject to the condition 

(5) Ej(X)sa forall 6€89,. 

The same difficulty now arises that presented itself in the general discussion 


of Chapter 1. Typically, the test that maximizes the power against a 
Particular alternative in K depends on this alternative, so that some 
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additional principle has to be introduced to define what is meant by an 
optimum test. There is one important exception: if K contains only one 
distribution, that is, if one is concerned with a single alternative, the 
problem is completely specified by (4) and (5). It then reduces to the 
mathematical problem of maximizing an integral subject to certain side 
conditions. The theory of this problem, and its statistical applications, 
constitutes the principal subject of the present chapter. In special cases it 
may of course turn out that the same test maximizes the power for all 
alternatives in K even when there is more than one. Examples of such 
uniformly most powerful (UMP) tests will be given in Sections 3 and 7. 

In the above formulation the problem can be considered as a special case 
of the general decision problem with two types of losses. Corresponding to 
the two kinds of error, one can introduce the two component loss functions, 


L,(8,d,)=lor0 as 0EQ, or8€Q,, 
L,(0, dy) =0 for all 0 


and 
L,(9,dy)=Oorl as 0EQ, or BE Q,, 
L,(0,d,) =0 for all 0. 


With this definition the minimization of EL,(8, 8(X)) subject to the 
restriction EL,(0, 8(X)) < a is exactly equivalent to the problem of hy- 
pothesis testing as given above. 

The formal loss functions L, and L, clearly do not represent in general 
the true losses. The loss resulting from an incorrect acceptance of the 
hypothesis, for example, will not be the same for all alternatives. The more 
the alternative differs from the hypothesis, the more serious are the conse- 
quences of such an error. As was discussed earlier, we have purposely 
forgone the more detailed approach implied by this criticism. Rather than 
working with a loss function which in practice one does not know, it seems 
preferable to base the theory on the simpler and intuitively appealing notion 
of error. It will be seen later that at least some of the results can be justified 
also in the more elaborate formulation. 


2. THE NEYMAN-PEARSON FUNDAMENTAL LEMMA 


A class of distributions is called simple if it contains only a single distribu- 
tion, and otherwise is said to be composite. The problem of hypothesis 


testing is completely specified by (4) and (5) if K is simple. Its solution is 
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easiest and can be given explicitly when the same is true of H. Let the 
distributions under a simple hypothesis H and alternative K be P; and P}, 
and suppose for a moment that these distributions are discrete with P,( X = 
x} = P(x) for i = 0,1. If at first one restricts attention to nonrandomized 
tests, the optimum test is defined as the critical region S satisfying 


(6) X P(x) <a 


xes 
and 


Y, P(x) = maximum. 
xesS 


It is easy to see which points should be included in S. To each point are 
attached two values, its probability under P) and under P,. The selected 
points are to have a total value not exceeding a on the one scale, and as 
large as possible on the other. This is a situation that occurs in many 
contexts. A buyer with a limited budget who wants to get “the most for his 
money" will rate the items according to their value per dollar. In order to 
travel a given distance in the shortest possible time, one must choose the 
speediest mode of transportation, that is, the one that yields the largest 
number of miles per hour. Analogously in the present problem the most 
valuable points x are those with the highest value of 


P,(x) 
r(x) e Po(x) | 


The points are therefore rated according to the value of this ratio and 
selected for S in this order, as many as one can afford under restriction (6). 
Formally this means that S is the set of all points x for which r(x) > c, 
where c is determined by the condition 


P{XES}= X Pl(x)=a. 


xir(x)»c 


Here a difficulty is seen to arise. It may happen that when a certain point is 
included, the value a has not yet been reached but that it would be 
exceeded if the next point were also included. The exact value a can then 
either not be achieved at all, or it can be attained only by breaking the 
preference order established by r(x). The resulting optimization problem 
has no explicit solution. (Algorithms for obtaining the maximizing set S are 
given by the theory of linear programming.) The difficulty can be avoided, 
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however, by a modification which does not require violation of the r-order 
and which does lead to a simple explicit solution, namely by permitting 
randomization.* This makes it possible to split the next point, including 
only a portion of it, and thereby to obtain the exact value a without 
breaking the order of preference that has been established for inclusion of 
the various sample points. These considerations are formalized in the 
following theorem, the fundamental lemma of Neyman and Pearson. 


Theorem 1. Let P, and P, be probability distributions possessing densities 
Po and p, respectively with respect to a measure pt 


(i) Existence. For testing H: pọ against the alternative K: p, there 
exists a test $ and a constant k such that 


(7) Eg(X) =a 
and 
1 when p,(x) > kpo(x), 
8 x)= i 
8) o(x) P when p,(x) < kpo(x). 


(ii) Sufficient condition for a most powerful test. Jf a test satisfies 
(7) and (8) for some k, then it is most powerful for testing py against p, at 
level a. 

(iii) Necessary condition for a most powerful test. If $ is most power- 
ful at level a for testing py against p,, then for some k it satisfies (8) a.e. p. It 
also satisfies (7) unless there exists a test of size < a and with power 1. 


Proof. For a — 0 and a — 1 the theorem is easily seen to be true 
provided the value k = + oo is admitted in (8) and 0 - oo is interpreted as 0. 
Throughout the proof we shall therefore assume 0 < a < 1. 

G): Let a(c) = Po p(X) > cpo(X)}. Since the probability is computed 
under Pp, the inequality need be considered only for the set where po(x) > 0, 
so that a(c) is the probability that the random variable piCX)/poCX 
Mus Eee 1 B a(c) is a cumulative distribution function, and a(c) is 

ii and continuous on the right, a(c — 0) — a(c) = 
Po{ P(X) /po(X) = c), a(— oo) = 1, and KO e 0. (MR e 0« : h 1, 
let c, be such that a(c;) < a < a(cg — 0), and consider the test defined 


*In practice, typically neither the breaking of the r-order nor randomization is considered 
acceptable. The common solution, instead, is to adopt a value of a that can be attained exactly 
and therefore does not present this problem. 


* There is no loss of generality in this assumption, since one can take p = P) + Py. 
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1 when p(x) > copo(x), 

y a — a(co) y 
$(x) Lu alco "s 0) E a(co) when P(x) Gi copo(x), 
0 when p(x) < copo(x). 


Here the middle expression is meaningful unless a(cy) = a(cọ — 0); since 
then Po{ p\(X) = copo( X)) = 0, ¢ is defined a.e. The size of is 


HO, | e ae (ea a} = 


so that cy can be taken as the k of the theorem. 

It is of interest to note that cy is essentially unique. The only exception is 
the case that an interval of c's exists for which a(c) = a. If (c’, c”) is such 
an interval, and 


E(X) = ^| 


rai i ptg 
C= (x: pols) > 0andc E ) 


then P,(C) = a(c’) — a(c" — 0) = 0. By Problem 3 of Chapter 2, this 
implies u(C) = 0 and hence P,(C) = 0. Thus the sets corresponding to two 
different values of c differ only in a set of points which has probability 0 
under both distributions, that is, points that could be excluded from the 
sample space. 

(ii): Suppose that $ is a test satisfying (7) and (8) and that $* is any 
other test with Ej$*(X) < a. Denote by S* and S~ the sets in the sample 
space where (x) — ó*(x) > 0 and < 0 respectively. If x is in S*, $(x) 
must be > 0 and p,(x) > kpo(x). In the same way p;(x) < kpo(x) for all 
x in S~, and hence 


J(e- oi ~ kpo) di =f. (67 9n — kpo) dp = 0. 
S*us 
The difference in power between $ and $* therefore satisfies 
fie- 9m du = k fo = 4) podu = 0, 


as was to be proved. 
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(iii): Let $* be most powerful at level a for testing po against p,, and let 
9 satisfy (7) and (8). Let S be the intersection of the set StU S^, on which 
p and ¢* differ, with the set (x: p,(x) * kpo(x)}, and suppose that 
a(S) » 0. Since (6 — ¢*)( p; — kpo) is positive on S, it follows from 
Problem 3 of Chapter 2 that 


f (6 — $*)( pi — kpo) du = f ($ — $*)(p1 - kpo) dy > 0 
S+US— s 


and hence that is more powerful against p, than $*. This is a contradic- 
tion, and therefore a(S) = 0, as was to be proved. 

If $* were of size < a and power < 1, it would be possible to include in 
the rejection region additional points or portions of points and thereby to 
increase the power until either the power is 1 or the size is a. Thus either 
Eyp*(X) = a or E,o*(X) = 1. 

The proof of part (iii) shows that the most powerful test is uniquely 
determined by (7) and (8) except on the set on which p,(x) = kpo(x). On 
this set, can be defined arbitrarily provided the resulting test has size «. 
Actually, we have shown that it is always possible to define $ to be constant 
over this boundary set. In the trivial case that there exists a test of power 1, 
the constant k of (8) is 0, and one will accept H for all points for which 
pix) = kpo(x) even though the test may then have size < a. 

It follows from these remarks that the most powerful test is determined 
uniquely (up to sets of measure zero) by (7) and (8) whenever the set on 
which p,(x) = kpo(x) has p-measure zero. This unique test is then clearly 
nonrandomized. More generally, it is seen that randomization is not re- 
quired except possibly on the boundary set, where it may be necessary to 
randomize in order to get the size equal to a. When there exists a test of 
power 1, (7) and (8) will determine a most powerful test, but it may not be 


unique in that there may exist a test also most powerful and satisfying (7) 
and (8) for some a’ < a. 


Corollary 1. Let B denote the power of the most powerful level- test 
(0 < a <1) for testing Py against P,. Then a < B unless Py = P}. 


Proof. Since the level-a test given by #(x) = a has power a, it is seen 
that a < B. If a — < 1, the test (x) =a is most powerful and by 
bike 1(iii) must satisfy (8). Then py(x) = p,(x) a.e. p, and hence 
P, = Py. 

An alternative method for proving the results of this section is based on 
the following geometric representation of the problem of testing a simple 
hypothesis against a simple alternative. Let N be the set of all points (a, £) 


3.2] THE NEYMAN-PEARSON FUNDAMENTAL LEMMA : 717 


for which there exists a test $ such that 
a=E6(X),  B-EQ(X). 


This set is convex, contains the points (0,0) and (1,1), and is symmetric 
with respect to the point (4, 4) in the sense that with any point (a, £) it also 
contains the point (1 — a,1 — f). In addition, the set N is closed. [This 
follows from the weak compactness theorem for critical functions, Theorem 
3 of the Appendix; the argument is the same as that in the proof of 
Theorem 5(i).] 

For each value 0 < a < 1, the level-ay tests are represented by the 
points whose abscissa is < ay. The most powerful of these tests (whose 
existence follows from the fact that N is closed) corresponds to the point on 
the upper boundary of N with abscissa ay. This is the only point corre- 
sponding to a most powerful level-a, test unless there exists a point (a, 1) in 
N with a < a, (Figure 1b). 

As an example of this geometric approach, consider the following alter- 
native proof of Corollary 1. Suppose that for some 0 « a, « 1 the power of 
the most powerful level-a test is a. Then it follows from the convexity of 
N that (a, 8) € N implies B < a, and hence from the symmetry of N that 
N consists exactly of the line segment connecting the points (0,0) and (1, 1). 
This means that {po du = fọp, dp for all $ and hence that po = p, (a.e. 
11), as was to be proved. A proof of Theorem 1 along these lines is given in a 
more general setting in the proof of Theorem 5. 

The Neyman-Pearson lemma has been generalized in many directions. 
An extension to the case of several side conditions is given in Section 6, and 
this result is further generalized in Section 8. A sequential version, due to 
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Wald and Wolfowitz (1948, 1950), plays a fundamental role in sequential 
analysis [see, for example, Ghosh (1970)]. Extensions to stochastic processes 
are discussed by Grenander (1950) and Dvoretzky, Kiefer, and Wolfowitz 
(1953), and a version for abstract spaces by Grenander (1981, Section 3.1). 
A modification due to Huber, in which the distributions are known only 
approximately, is presented in Section 3 of Chapter 9. 

An extension to a selection problem, proposed by Birnbaum and 
Chapman (1950), is sketched in Problem 23. Generalizations to a variety of 
decision problems with a finite number of actions can be found, for 
example, in Hoel and Peterson (1949), Karlin and Rubin (1956), Karlin and 
Truax (1960), Lehmann (1961), Hall and Kudo (1968) and Spjetvoll (1972). 


3. DISTRIBUTIONS WITH MONOTONE 
LIKELIHOOD RATIO 


The case that both the hypothesis and the class of alternatives are simple is 
mainly of theoretical interest, since problems arising in applications typi- 
cally involve a parametric family of distributions depending on one or more 
parameters. In the simplest situation of this kind the distributions depend 
on a single real-valued parameter 0, and the hypothesis is one-sided, say 
H: 0 < 6). In general, the most powerful test of H against an alternative 
0, > 6, depends on @, and is then not UMP. However, a UMP test does 
exist if an additional assumption is satisfied. The real-parameter family of 
densities p(x) is said to have monotone likelihood ratio* if there exists a 
real-valued function T(x) such that for any 0 < 6" the distributions P; and 


5 ^M distinct, and the ratio pj (x)/ps(x) is a nondecreasing function of 
x). 


Theorem 2. Let 0 be a real parameter, and let the random variable X 
have probability density p(x) with monotone likelihood ratio in T(x). 


6) For testing H: 0 < 0, against K : 0 > Oo, there exists a UMP test, 
which is given by 


1 whe T(x)»C, 
(9) $(x) "(v when T(x)- C, 
0 when T(x)«C, 


| "is misc is p oe of specific versions of the densities py. If instead the definition is 
o be given in terms of the distributions P}, various null-set considerati i 
densi ia Pascal ae nsiderations enter which are 
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where C and y are determined by 
(10) E,g(X) =a. 
(ii) The power function 
B(8) = E(X) 


of this test is strictly increasing for all points 0 for which 0 < B(0) < 1. 

(iii) For all 0’, the test determined by (9) and (10) is UMP for testing 
H’: 0 < 0' against K’: 0 > 0' at level a’ = B(0"). 

(iv) For any 0 < 6, the test minimizes B(@) (the probability of an error of 
the first kind) among all tests satisfying (10). 


Proof. (i) and (ii): Consider first the hypothesis Hy: 0 = 0, and some 
simple alternative 8, > 4. The most desirable points for rejection are those 
for which r(x) = ps (x)/Pe (x) = g[T(x)] is sufficiently large. If T(x) < 
T(x’), then r(x) < r(x’) and x’ is at least as desirable as x. Thus the test 
which rejects for large values of T(x) is most powerful. As in the proof of 
Theorem 1(i), it is seen that there exist C and y such that (9) and (10) hold. 
By Theorem 1(ii), the resulting test is also most powerful for testing Py, 
against Pj, at level a’ = B(0") provided 8’ < 6”. Part (ii) of the present 
theorem now follows from Corollary 1. Since B(@) is therefore nondecreas- 
ing, the test satisfies 


(11) Eyo(X) <a for 6&6, 


The class of tests satisfying (11) is contained in the class satisfying Ej $( X) 
< a. Since the given test maximizes B(0,) within this wider class, it also 
maximizes (6,) subject to (11); since it is independent of the particular 
alternative 0, > 6, chosen, it is UMP against K. 

(iii) is proved by an analogous argument. 

(iv) follows from the fact that the test which minimizes the power for 
testing a simple hypothesis against a simple alternative is obtained by 
applying the fundamental lemma (Theorem 1) with all inequalities reversed. 

By interchanging inequalities throughout, one obtains in an obvious 
manner the solution of the dual problem, H: 0 > 6, K: 0 < 6. 

The proof of (i) and (ii) exhibits the basic property of families with 
monotone likelihood ratio: every pair of parameter values 6) < 0, estab- 
lishes essentially the same preference order of the sample points (in the 
sense of the preceding section). A few examples of such families, and hence 
of UMP one-sided tests, will be given below. However, the main appli- 
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cations of Theorem 2 will come later, when such families appear as the set 
of conditional distributions given a sufficient statistic (Chapters 4 and 5) 
and as distributions of a maximal invariant (Chapters 6, 7, and 8). 


Example 1. Hypergeometric. From a lot containing N items of a manufac- 
tured product, a sample of size n is selected at random, and each item in the sample 
is inspected. If the total number of defective items in the lot is D, the number X of 
defectives found in the sample has the Aypergeometric distribution 


D\(N-D 
P{X=x} no oh 


Interpreting Pj (x) as a density with respect to the measure p that assigns to any set 
on the real line as measure the number of integers 0,1,2,... that it contains, and 
noting that for values of x within its range 


, max(0,n+ D= N) <x < min(n, D). 


Py Ax) ca este Maa Del 
yy eoim be 
0 or oo if x=n+D—-NorD+1, 


if n+D+1-N<x<D, 


it is seen that the distributions satisfy the assumption of monotone likelihood ratios 
with T(x) = x. Therefore there exists a UMP test for testing the hypothesis 
H: D < Dy against K: D > Dy, which rejects H when X is too large, and an 
analogous test for testing H’: D > Dy. 


$ An important class of families of distributions that satisfy the assump- 
tions of Theorem 2 are the one-parameter exponential families. 


Corollary 2. Let 8 be a real parameter, and let X have probability density 
(with respect to some measure p) 


(12) Po(x) = C(8)e9 ron (x), 


where Q is strictly monotone. Then there exists a UMP test @ for testing 
H: 0 < 6, against K: 0 > &,. If Q is increasing, 


$(x)=1,7,0 as T(x) *, 2, «C, 


where C and y are determined by E(X) =a. If Q is decreasing, the 
inequalities are reversed. 

A converse of Corollary 2 is given by Pfanzag] (1968), who shows under 
weak regularity conditions that the existence of UMP tests against one-sided 
alternatives for all sample sizes and one value of « imp! 


li ial 
family. es an exponentia 
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As in Example 1, we shall denote the right-hand side of (12) by P,(x) 
instead of pg(x) when it is a probability, that is, when X is discrete and p is 
counting measure. 


Example 2. Binomial. The binomial distributions b( p, n) with 
P) -(Z)ra = p)"* 


satisfy (12) with T(x) = x, 0 — p, Q(p) = log[ p/(1 — p)]. The problem of testing 
H p > po arises, for instance, in the situation of Example 1 if one supposes that 
the production process is in statistical control, so that the various items constitute 
independent trials with constant probability p of being defective. The number of 
defectives X in a sample of size n is then a sufficient statistic for the distribution of 
the variables X, (i = 1,..., n), where X, is 1 or 0 as the ith item drawn is defective 
or not, and X is distributed as b( p, n). There exists therefore a UMP test of H, 
which rejects H when X is too small. ' 

An alternative sampling plan which is sometimes used in binomial situations is 
inverse binomial sampling. Here the experiment is continued until a specified number 
m of successes—for example, cures effected by some new medical treatment—have 
been obtained. If Y, denotes the number of trials after the (i — 1)st success up to 
but not including the ith success, the probability that Y, = y is pq” for y = 0,1,..., 
so that the joint distribution of Y;,..., Yẹ is 


P, Ym) = PGE", My =0,1,..., k-1,...,m. 


This is an exponential family with T(y) = Ly, and Q(p) = log(1 — p). Since 
Q(p) is a decreasing function of p, the UMP test of H: p < p, rejects H when T 
is too small. This is what one would expect, since the realization of m successes in 
only a few more than m trials indicates a high value of p. The test statistic T, which 
is the number of trials required in excess of m to get m successes, has the negative 
binomial distribution [Chapter 1, Problem 1(i)] 


p(y - ("t t pone, t=0,1,.... 


Example 3. Poisson. If X,,..., X, are independent Poisson variables with 
E(X,) =X, their joint distribution is 


yat» 
ED 
AS roni) 

P. (35:5 Xn) Peale x,! 


This constitutes an exponential family with T(x) = Xx;, and Q(A) = log À. One- 
sided hypotheses concerning À might arise if À is a bacterial density and the X’s 
are a number of bacterial counts, or if the X’s denote the number of a-particles 
produced in equal time intervals by a radioactive substance, etc. The UMP test of 
the hypothesis A < Ay rejects when LX; is too large. Here the test statistic Z X; has 
itself a Poisson distribution with parameter nA. 
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Instead of observing the radioactive material for given time periods or counting 
the number of bacteria in given areas of a slide, one can adopt an inverse sampling 
method. The experiment is then continued, or the area over which the bacteria are 
counted is enlarged, until a count of m has been obtained. The observations consist 
of the times 7,,...,7,, that it takes for the first occurrence, from the first to the 
second, and so on. If one is dealing with a Poisson process and the number of 
occurrences in a time or space interval 7 has the distribution 


(Ar)* 


X 
En "EOM xima e, 


P(x) = 


then the observed times are independently distributed, each with the exponential 
probability density A e~*’ for t > 0 [Problem 1(ii) of Chapter 1]. The joint densities 


TEES THUS) -xe(AE:). fps Lestat 0, 


i=l 


form an exponential family with T(1,...,7,,) = Xt, and Q(A) = —A. The UMP 
test of H: A < A, rejects when T = YT, is too small. Since 27, has density }e~"/” 
for u > 0, which is the density of a x?-distribution with 2 degrees of freedom, 2AT 
has a x?-distribution with 2m degrees of freedom. The boundary of the rejection 
region can therefore be determined from a table of x?. 


The formulation of the problem of hypothesis testing given at the 
beginning of the chapter takes account of the losses resulting from wrong 
decisions only in terms of the two types of error. To obtain a more detailed 
description of the problem of testing H: 0 < 6, against the alternatives 
0 > 6), one can consider it as a decision problem with the decisions dọ and 
d, of accepting and rejecting H and a loss function L(6, d;) = L,(0). 
Typically, L;(8) will be 0 for 0 < 6, and strictly increasing for 0 > 6,, and 
L,(4) will be strictly decreasing for 0 < 6, and equal to 0 for 0 > 6,. The 
difference then satisfies 


(13) L(0)-L,(0)Z0 as 0S6, 


The following theorem is a special case of complete class results of Karlin 
and Rubin (1956) and Brown, Cohen, and Strawderman (1976). 


Theorem 3. 


(i) Under the assumptions of Theorem 2, the family of tests given by (9) 
and (10) with 0 < a < 1 is essentially complete provided the loss function 
satisfies (13). 

(i) This family is also minimal essentially complete if the set of points x 
for which pg(x) > 0 is independent of 0. 
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Proof. A (i): The risk function of any test $ is 
R(6,9) = fpo(x){o(x)Li(4) + [1 -= 6(x)] Lo(8)} dnx) 


= fro(x){Lo(8) + [L1(8) -= Lo(4)]o(x)} du(x), 
and hence the difference of two risk functions is 
R(0, 4) — R(0, $) = [L.(6) - Lo(8)] f(& — $) po dn. 
This is x 0 for all 0 if 


By(8) - B(0) = f(e —o)mdnO for 62. 


Given any test $, let Ej$(X) = a. It follows from Theorem 2(i) that there 
exists a UMP level-a test ¢ for testing 0 = 6; against 6 > ĝo, which 
satisfies (9) and (10). By Theorem 2(iv), 4 also minimizes the power for 
0 < 6,. Thus the two risk functions satisfy R(0, d/) < R(0, >) for all 6, as 
was to be proved. i 

(ii): Let 6, and y be of sizes a < a’ and UMP for testing 6, against 
0 > 6. Then B, (8) < B,, (0) for all @ > 6, unless B, (0) = 1. By consider- 
ing the problem of testing 0 = 6; against 0 < 6, it is seen analogously that 
this inequality also holds for all 0 < 6 unless f, (0) = 0. Since the 
exceptional possibilities are excluded by the assumptions, it follows that 
R(O, #’) € R(0, $) as 0 Z Oy. Hence each of the two risk functions is better 
than the other for some values of 6. 

The class of tests previously derived as UMP at the various significance 
levels æ is now seen to constitute an essentially complete class for a much 
more general decision problem, in which the loss function is only required 
to satisfy certain broad qualitative conditions. From this point of view, the 
formulation involving the specification of a level of significance can be 
considered as a simple way of selecting a particular procedure from an 
essentially complete family. ^ 

The property of monotone likelihood ratio defines a very strong ordering 
of a family of distributions. For later use, we consider also the following 
somewhat weaker definition. A family of cumulative distribution functions 
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F on the real line is said to be stochastically increasing (and the same term 
is applied to random variables possessing these distributions) if the distribu- 
tions are distinct and if 0 < 0^ implies Fj(x) 2 Fj(x) for all x. If then 
X and X’ have distributions F} and Fj respectively, it follows that 
P{X > x) < P( X' » x) for all x, so that X’ tends to have larger values 
than X. In this case the variable X’ is said to be stochastically larger than 
X. This relationship is made more intuitive by the following characterization 
of the stochastic ordering of two distributions. 


Lemma 1. Let F) and F, be two cumulative distribution functions on the 
real line. Then F,(x) € K(x) for all x if and only if there exist two 
nondecreasing functions f, and fj, and a random variable V, such that (a) 
fo(v) < f,(v) for all v, and (b) the distributions of f (V) and f(V) are Fy 
and F, respectively. 


Proof. Suppose first that the required fj, f, and V exist. Then 
F(x) = P(f(V) <x} < P{f,(V) < x) = R(x) 


for all x. Conversely, suppose that F,(x) < F(x) for all x, and let 
f(y) = inf(x: F(x -0) < y < F(x)), i = 0,1. These functions are non- 
decreasing and for f, — f, F, — F satisfy 


i 


flF(x) sx and F[f(y)) >y forall x and y. 


It follows that y < F(x) implies f(y) < f[F(xo)] € xy and that con- 
versely f(y) < Xo implies F[f(y)] < F(x) and hence y < F(x), so that 
the two inequalities f(y) € xy and y < F(x,) are equivalent. Let V be 
uniformly distributed on (0,1). Then P(f(V)xx)- P(V < F(x)) 
= F(x). Since F,(x) < F(x) for all x implies fo(y) < fiCy) for all y, this 
completes the proof. 


One of the simplest examples of a stochastically ordered family is a 
location parameter family, that is, a family satisfying 


R(x) = F(x — 0). 


To see that this is stochastically increasing, let X be a random variable with 
distribution F(x). Then 0 < 6’ implies 


F(x — 0) = P{X<x-0}>P{X<x- 6} - F(x — 6^), 


as was to be shown. 
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Another example is furnished by families with monotone likelihood ratio. 
This is seen from the following lemma, which establishes some basic 
properties of these families. 


Lemma 2. Let pg(x) be a family of densities on the real line with 
monotone likelihood ratio in x. 

(i) If y is a nondecreasing function of x, then EJ ( X) is a nondecreasing 
function of 0; if X,,..., X, are independently distributed with density pg and 
y is a function of x,,..., x, which is nondecreasing in each of its arguments, 
then EgJ/( X,,..., X,) is a nondecreasing function of 0. 

(ii) For any 0 < 0', the cumulative distribution functions of X under 0 
and 0' satisfy 


Fy (x) < F(x) forall x. 


(iii) Let V be a function with a single change of sign. More specifically, 
suppose there exists a value xo such that (x) < 0 for x < xo and (x) > 0 
for x> xg. Then there exists 0) such that Egj(X) <0 for 0 < 0, and 
Eg (X) = 0 for 8 > b, unless Eg (X) is either positive for all 0 or negative 
for all 0. 

(iv) Suppose that py(x) is positive for all 0 and all x, that Po(x)/po(x) 
is strictly increasing in x for 0 < 8’, and that (x) is as in (iii) and is #0 
with positive probability. If Ep(X) = 0, then Eg (X) < 0 for 0 < 0, and 
> 0 for 0 > 6,. 

Proof. (i): Let 0 < 6’, and let A and B be the sets for which pg(x) < 
Po(x) and p(x) > po(x) respectively. If a = sup,y(x) and b = inf jJ (x), 
then b — a > 0 and 


[vee - po) du > a f (py — po) du + b f (po = Po) dn 


= (b — a) f (pe — Po) du = 0, 


which proves the first assertion. The result for general n follows by 
induction. 

(ii); This follows from (i) by letting (x) = 1 for x > xo and y(x) = 0 
otherwise. 

(ii): We shall show first that for any 0^ < 0", Ee} (X) > 0 implies 
Eg(X) > 0. If pys(xo)/Pe(xo) = 90, then pe(x) = 0 for x > xo and 
hence Ey(X) <0. Suppose therefore that Por(X0)/Po(Xo) = c « oo. 
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Then w(x) > 0 on the set S = (x: pe(x) = 0 and py(x) > 0}, and 
Por 
Epy (X) = | V— Po di 
wa CX) fy Pe pe ap 


= f me du + f cy pg du = cE gy (X) > 0. 


The result now follows by letting 0) = inf{@: Eg (X) > 0). 

(iv): The proof is analogous to that of (iii). 

Part (ii) of the lemma shows that any family of distributions with 
monotone likelihood ratio in x is stochastically increasing. That the con- 
verse does not hold is shown for example by the Cauchy densities 


1 1 
T14(x—-80)' 


The family is stochastically increasing, since @ is a location parameter; 
however, the likelihood ratio is not monotone. Conditions under which a 
location parameter family possesses monotone likelihood ratio are given in 
Chapter 9, Example 1. 

Lemma 2 is a special case of a theorem of Karlin (1957, 1968) relating 
the number of sign changes of Ej ( X) to those of y(x) when the densities 
Pa(x) are totally positive (defined in Problem 27). The application of totally 
positive—or equivalently, variation diminishing— distributions to statistics 


is discussed by Brown, Johnstone, and MacGibbon (1981); see also Problem 
30. 


4. COMPARISON OF EXPERIMENTS* 


Suppose that different experiments are available for testing a simple hy- 
pothesis H against a simple alternative K. One experiment results in a 
random variable X, which has probability densities f and g under H and K 
respectively; the other leads to the observation of X’ with densities f’ and 
g’. Let B(a) and f'(a) denote the power of the most powerful level-a test 
based on X and X’. In general, the relationship between (a) and f/(a) 
will depend on a. However, if B’(a) < (a) for all a, then X or the 
experiment ( f, g) is said to be more informative than X’. As an example, 
suppose that the family of densities p(x) is the exponential family (12) and 


*This section constitutes a digression and may be omitted. 
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that f= f’ = Pop Z= Pop ' = po, where 0 < 0, « 6. Then (f, g) is 
more informative than ( f’, g^) by Theorem 2. 

A simple sufficient condition* for X to be more informative than X’ is 
the existence of a function A(x, u) and a random quantity U, independent 
of X and having a known distribution, such that the density of Y = A(X, U) 
is f’ or g’ as that of X is f or g. This follows, as in the theory of sufficient 
statistics, from the fact that one can then construct from X (with the help of 
U) a variable Y which is equivalent to X’. One can also argue more 
specifically that if @(x’) is the most powerful level-a test for testing f’ 
against g’ and if y(x) = Eọ[h(x, U)], then EJ(X) = E$( X^) under both 
H and K. The test W(x) is therefore a level-a test with power B’(a), and 
hence B(a) > B’(a). 

When such a transformation A exists, the experiment ( f, g) is said to be 
sufficient for ( f’, g^). If then X,,..., X, and X/,..., X; are samples from 
X and X’ respectively, the first of these samples is more informative than 
the second one. It is also more informative than (Z,,..., Z,) where each Z, 
is either X, or X/ with certain probabilities. 


Example 4. 2 x 2 Table. Two characteristics A and B, which each member of 
a population may or may not possess, are to be tested for independence. The 
probabilities p = P(A) and m = P(B), that is, the proportions of individuals 
possessing properties A and B, are assumed to be known. This might be the case, 
for example, if the characteristics have previously been studied separately but not in 
conjunction. The probabilities of the four possible combinations AB, AB, AB, and 
AB under the hypothesis of independence and under the alternative that P(AB) has 
a specified value p are 


Under H: Under K: 


B B B B 


A pr p(l- 7) p p-p 
A Q- p) ( - py - 7) T-p l-p-mT*p 


The experimental material is to consist of a sample of size s. This can be selected, 
for example, at random from those members of the population possessing property 
A. One then observes for each member of the sample whether or not it possesses 
property B, and hence is dealing with a sample from a binomial distribution with 
probabilities 


P 


H: P(B\|A) =7 and K: P(BIA) = a 


Alternatively, one can draw the sample from one of the other categories B, B, or A, 


*For a proof that this condition is also necessary see Blackwell (1951b). 
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obtaining in each case a sample from a binomial distribution with probabilities 
given by the following table: 


Population 
Sampled Probability H K 
A P(B|A) 7 p/p 
B P(A|B) p p/" 
B P(A|B) P (p - 9)/ - 7) 
A P(B|A) 7 (7 — p)/Q — p) 


Without loss of generality let the categories A, A, B, and B be labeled so that 
p <7 x<}. We shall now show that of the four experiments, which consist in 
Observing an individual from one of the four categories, the first one (sampling from 
A) is most informative and in fact is sufficient for each of the others. 

To compare A with B, let X and X’ be 1 or 0, and let the probabilities of their 
being equal to 1 be given by the first and the second row of the table respectively. 
Let U be uniformly distributed on (0,1) and independent of X, and let Y — 
h(X,U) = 1 when X = 1 and U < p/m, and Y = 0 otherwise. Then P(Y-1)isp 
under H and p/m under K, so that Y has the same distribution as X". This proves 
that X is sufficient for X’, and hence is the more informative of the two. For the 
comparison of A with B define Y to be 1 when X = 0 and U < p/(1 — 7), and to 
be 0 otherwise. Then the probability that Y — 1 coincides with the third row of the 
table. Finally, the probability that Y — 1 is given by the last row of the table if one 
defines Y to be equal to 1 when X = 1 and U < (7 — p)/(1 — p) and when X = 0 
and U > (1— 7 — p)/(1 — p). 

It follows from the general remarks preceding the example that if the experimen- 
tal material is to consist of s individuals, these should be drawn from category A, 
that is, the rarest of the four categories, in preference to any of the others. This is 
preferable also to drawing the s from the population at large, since the latter 
procedure is equivalent to drawing each of them from either A or A with probabili- 
ties p and 1 — p respectively. 

The comparison between these various experiments is independent not only of a 
but also of p. Furthermore, if a sample is taken from A, there exists by Corollary 2 


P(BIA) > 7 and hence p > pz, according to which the probabilities of AB and AB 
are larger, and those of AB and AB smaller, than under the assumption of 
independence. This test therefore provides the best power that can be obtained for 
the hypothesis of independence on the basis of a sample of size s. 


against A, for these distributions arises also for 
where one is concerned with the number of particl 


in an obvious sense is seen from the fact that the unique most powerful test for 
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testing Ag against A, depends on X + Y and therefore cannot be duplicated from 
X alone. 

Sometimes it is not possible to count the number of occurrences but only to 
determine whether or not at least one event has taken place. In the dilution method 
in bacteriology, for example, a bacterial culture is diluted in a certain volume of 
water, from which a number of samples of fixed size are taken and tested for the 
presence or absence of bacteria. In general, one observes then for each of n intervals 
whether an event occurred. The result is a binomial variable with probability of 
success (at least one occurrence) 


p=l-e™. 


Since a very large or small interval leads to nearly certain success or failure, one 
might suspect that for testing Ag against A, intermediate values of v would be more 
informative than extreme ones. However, it turns out that the experiments (Av, A,v) 
and (Aow, Àw) are not comparable for any values of v and w. (See Problem 19.) 
For a discussion of how to select v in this and similar situations see Hodges (1949). 


The definition of an experiment @ being more informative than an 
experiment &’ can be extended in a natural way to probability models 
containing more than two distributions by requiring that for any decision 
problem a risk function that is obtainable on the basis of £’ can be 
matched or improved upon by one based on &. Unfortunately, interesting 
pairs of experiments permitting such a strong ordering are rare. (For an 
example, see Problems 11 and 12 of Chapter 7). LeCam (1964) initiated a 
more generally applicable method of comparison by defining a measure of 
the extent to which one experiment is more informative than another. A 
survey of some of the principal concepts and results of this theory is given 
by Torgersen (1976). 


5. CONFIDENCE BOUNDS 


The theory of UMP one-sided tests can be applied to the problem of 
obtaining a lower or upper bound for a real-valued parameter 0. The 
problem of setting a lower bound arises, for example, when @ is the 
breaking strength of a new alloy; that of setting an upper bound, when @ is 
the toxicity of a drug or the probability of an undesirable event. The 
discussion of lower and upper bounds is completely parallel, and it is 


therefore enough to consider the case of a lower bound, say 9. 
Since @ = (X) will be a function of the observations, it cannot be 


required to fall below 6 with certainty, but only with specified high 
probability. One selects a number 1 — a, the confidence level, and restricts 
attention to bounds 6 satisfying 


(14) P(&(X)s0)21-« forall ð. 
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The function @ is called a lower confidence bound for 0 at confidence level 
1 — a; the infimum of the left-hand side of (14), which in practice will be 
equal to 1 — a, is called the confidence coefficient of 8. 

Subject to (14), @ should underestimate @ by as little as possible. One can 
ask, for example, that the probability of @ falling below any 0^ « @ should 
be a minimum. A function @ for which 


(15) P,{9(X) < 6’} = minimum 


for all 0^ < 0 subject to (14) is a uniformly most accurate lower confidence 
bound for @ at confidence level 1 — a. 

Let L(0, 0) be a measure of the loss resulting from underestimating 0, so 
that for each fixed @ the function L(8, 8) is defined and nonnegative for 
8 < 0, and is nonincreasing in its second argument. One would then wish to 
minimize 
(16) E,L(0, 0) 


subject to (14). It can be shown that a uniformly most accurate lower 
confidence bound @ minimizes (16) subject to (14) for every such loss 
function L. (See Problem 21.) 

The derivation of uniformly most accurate confidence bounds is facili- 
tated by introducing the following more general concept, which will be 
considered in more detail in Chapter 5. A family of subsets S(x) of the 


parameter space Q is said to constitute a family of confidence sets at 
confidence level 1 — a if 


(17) P(€s(X)) 21-a forall 0 EQ, 


that is, if the random set S(X) covers the true parameter point with 
probability > 1 — a. A lower confidence bound corresponds to the special 
case that S(x) is a one-sided interval 


S(x) = (6:0(x) «0 « o). 


Theorem 4. 

() For each 0, € Q let A 
testing H(05) : 0 = 6, 
parameter values 


(85) be the acceptance region of a level-a test for 
and for each sample point x let S(x) denote the set of 


S(x) = (6:x € 4(0),0 € q). 


Then S(x) is a family of confidence sets for 0 at confidence level 1 — a. 
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Gi) If for all 6), A(85) is UMP for testing H(@) at level a against the 
alternatives K(0,), then for each 6, in 2, S(X) minimizes the probability 
P,(6,€ S(X)) forall 6 € K(@) 


among all level -(1 — a) families of confidence sets for 0. 


Proof. (i): By definition of S(x), 
(18) 6€S(x) ifandonlyif x € A(6), 
and hence 
P,{0 € S(X)} = Py{ XE A(8)} 2 1—- a. 


(ii): If S*(x) is any other family of confidence sets at level 1 — a, and if 
A*(0) = (x:0 € S*(x)), then 


P(X € 4*(0)) = (665*()) 21- a, 


so that 4*(8,) is the acceptance region of a level-a test of H(6,). It follows 
from the assumed property of 4(65) that for any 0 € K(6,) 


P {X E A*(0)} = P,{ X € A(4)} 
and hence that 
P,(6,€ S*(X)} = Po{% © S(X)}, 


as was to be proved. 

The equivalence (18) shows the structure of the confidence sets S(x) as 
the totality of parameter values 0 for which the hypothesis H(0) is accepted 
when x is observed. A confidence set can therefore be viewed as a combined 
statement regarding the tests of the various hypotheses H(@), which exhibits 
the values for which the hypothesis is accepted [0 € S(x)] and those for 
which it is rejected [0 € S(x)]. 

Corollary 3. Let the family of densities p(x), 6 € Q, have monotone 
likelihood ratio in T(x), and suppose that the cumulative distribution function 
F,(t) of T = T(X) is a continuous function in each of the variables t and 0 
when the other is fixed. 

(i) There exists a uniformly most accurate confidence bound 8 for 0 at 
each confidence level 1 — a. 
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(i) If x denotes the observed values of X and t = T(x), and if the 
equation 


(19) A(t)=1-a 
has a solution 0 = 6 in Q, then this solution is unique and (x) = ê. 


Proof. (i): There exists for each 8, a constant C(6,) such that 
P {T> C(6))) = a, 


and by Theorem 2, T > C() is a UMP level-a rejection region for testing 
0 = 6, against 0 > 6,. By Corollary 1, the power of this test against any 
alternative 0, > 6, exceeds a, and hence C(6,) < C(@;) so that the function 
C is strictly increasing; it is also continuous. Let A()) denote the accep- 
tance region T < C(6,), and let S(x) be defined by (18). It follows from the 


monotonicity of the function C that S(x) consists of those values 6 € Q 
which satisfy 0 < 0, where 


8 = inf (0: T(x) < C(0)). 


By Theorem 4, the sets {9: 0(x) < 0), restricted to possible values of the 
parameter, thus constitute a family of confidence sets at level 1 — a, which 
minimize Py{8 < 0") for all 8 & K(6’), that is, for all 0 > 6’. This shows 8 
to be a uniformly most accurate confidence bound for 6. 

(ii): It follows from Corollary 1 that F(t) is a strictly decreasing 
function of @ at any point t for which 0 < F,(t) < 1, and hence that (19). 
can have at most one solution. Suppose now that £ is the observed value of 
T and that the equation F(t)=1-a has the solution à € Q. Then 
F(t) = 1 — a, and by definition of the function C, cô) = t. The inequality 
t < C(0) is then equivalent to c(6) < C(0) and hence to 6 < 6. It follows 
that 0 = 6, as was to be proved. 


: Under the same assumptions, the corresponding upper confidence bound 
with confidence coefficient 1 — a is the solution 8 of the equation P,{T 2 


t} = 1 — a or equivalently of F(t) = a. 


Example 6. Exponential waiting times. To determine an upper b 
. Exp i ound for the 
degree of radioactivity À of a radioactive substance, the wee is observed until 
" E € pren she ee a Geiger counter. Under the assumptions of 
5 joint prol ty density of the times T (i= i 
between the (i — 1)st count and the pde is es T,(i= 1,..., m) elapsing 


(h.t) m NPE, uo SO, 


If T — YT, denotes the total time of observation, then 2AT has a x?-distribution 
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with 2m degrees of freedom, and, as was shown in Example 3, the acceptance region 
of the most powerful test of H(Ao): A = Ao against A < Ay is 2AgT < C, where C 
is determined by the equation F 


Tis ric 


The set S(5,...,1,) defined by (18) is then the set of values A such that 
A < C/2T, and it follows from Theorem 4 that À = C/2T is a uniformly most 
accurate upper confidence bound for A. This result-can also be obtained through 
Corollary 3. 


If the variables X or T are discrete, Corollary 3 cannot be applied 
directly, since the distribution functions F(t) are not continuous, and for 
most values 0, the optimum tests of H : @ = 0, are randomized. However, 
any randomized test based on X has the- following representation as a 
nonrandomized test depending on X and an independent variable U 
distributed uniformly over (0,1). Given a-critical function $, consider the 
rejection region ; 


R= ((x, u): u < 6(x)}. 
Then , 
P{(X,U) € R) = P(U < $(X)} = £4(X), 


whatever the distribution of X, so that R has the same power function as $ 
and the two tests are equivalent. The pair of variables (X, U ) has a 
particularly simple representation when X is integer-valued. In this case the 
Statistic " 


T=X+U 
is equivalent to the pair (X, U), since with probability 1 
X= [T] $98 U Ec [T]; 

where [T] denotes the largest integer « T. The distribution of T is 
continuous, and confidence bounds can be based on this statistic. 

Example 7. Binomial. An upper bound is required for a binomial probability 
p—for example, the probability that a batch of polio vaccine manufactured accord- 
ing to a certain procedure contains any live virus. Let X,,..., X, denote the 


outcomes of n trials, X, being 1 or 0 with probabilities p and q respectively, and let 
X - XX, Then T = X + U has probability density 
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This satisfies the conditions of Corollary 3, and the upper confidence bound jp is 
therefore the solution, if it exists, of the equation 


P(T«1)-a, 


where t is the observed value of T. A solution does exist for all values a € t € n + a. 
For n + a < t, the hypothesis H( po): p = po is accepted against the alternatives 
P < po for all values of p, and hence p = 1. For ¢ < a, H( po) is rejected for all 
values of py and the confidence set S(t) is therefore empty. Consider instead the 
sets S*(1) which are equal to S(t) for t > a and which for t < a consist of the 
single point p — 0. They are also confidence sets at level 1 — a, since for all p, 


P,{ p € S*(T)} > P,{pES(T)}=1-a. 
On the other hand, P(pes*(T)- P,{ p' € S(T)) for all p' > 0 and hence 


P(p'es*(T)) =P{p'€S(T)} forall pp. 


is then somewhat larger than necessary; as a compensation it also gives a corre- 
spondingly higher probability of not falling below the true p. j NT 

References to tables for the confidence bounds and a careful discussion of 
Various approximations can be found in Hall (1982) and Blyth (1984). 

_ Let @ and ĝ be lower and upper bounds for @ with confidence coeffi- 
cients 1 — a, and 1 — a,, and suppose that (x) < A(x) for all x. This will 
be the case under the assumptions of Corollary 3 if a + a, < 1. The 
intervals (0, 0) are then confidence intervals for 0 with confidence coefficient 
l-a- 5; that is, th 


! ey contain the true parameter value with probability 
— a — a), since 


P(6s6s8) -1-a — a, for all 8. 


If 0 and @ are uniformly most accurate, they minimize E,1,(0, 8) and 
Es L.(0, 0) at their Tespective levels for any function L, that is nonincreas- 


ing in 8 for @ < 6 and for 8 > 8 and any L i ing in 8 
0 > and 0 for  < 6. Letting SE EE E EEA IT for 


L(0; 0,8) = 1,(6,8) + L,(0,8), 
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the intervals (8, 8) therefore minimize E,1(6; 6, 0) subject to 
P{O>O} <a,  B(8«0)sa. 
An example of such a loss function is 


6-0 if 0<6<8, 
L(0;0,0)=(0-6 if 0<9, 
0-0 if 8«0, 


which provides a natural measure of the accuracy of the intervals. Other 
possible measures are the actual length 6 — @ of the intervals, or, for 
example, a(0 — 0)? + b(0 — 0)?, which gives an indication of the distance 
of the two end points from the true value.* 

An important limiting case corresponds to the levels a; = a, = +. Under 
the assumptions of Corollary 3 and if the region of positive density is 
independent of @ so that tests of power 1 are impossible when a < 1, the 
upper and lower confidence bounds 0 and 6 coincide in this case. The 
common bound satisfies 


P,{9 <0} = (820) =}, 


and the estimate @ of 0 is therefore as likely to underestimate as to 
overestimate the true value. An estimate with this property is said to be 
median unbiased. (For the relation of this to other concepts of unbiasedness, 
see Chapter 1, Problem 3.) It follows from the above result for arbitrary a, 
and a, that among all median unbiased estimates, ð minimizes EL(0, 0) 
for any monotone loss function, that is, any loss function which for fixed 0 
has a minimum of 0 at 0 = 6 and is nondecreasing as 8 moves away from 0 
in either direction. By taking in particular L(0,0) = 0 when | — | < A 
and = 1 otherwise, it is seen that among all median unbiased estimates, 0 
minimizes the probability of differing from @ by more than any given 
amount; more generally it maximizes the probability 


P,{-A,<9-@<A,} 
for any A), A, > 0. f 
A more detailed assessment of the position of @ than that provided by 


confidence bounds or intervals corresponding to a fixed level y = 1 — a is 
obtained by stating confidence bounds for a number of levels, for example 


*Proposed by Wolfowitz (1950). 


96 UNIFORMLY MOST POWERFUL TESTS [3.6 


upper confidence bounds corresponding to values such as y = .05, .1, .25, .5, 
.75, 9, 95. These constitute a set of standard confidence bounds,* from 
which different specific intervals or bounds can be obtained in the obvious 
manner. 


6. A GENERALIZATION OF THE FUNDAMENTAL LEMMA 


The following is a useful extension of Theorem 1 to the case of more than 
one side condition. 


Theorem 5. Let fj...,f,4, be real-valued functions defined on a 
Euclidean space ¥ and integrable p, and suppose that for given constants 
€, ++ +5 Cm there exists a critical function à satisfying 


(20) folde = cn PzE.m. 


Let @ be the class of critical functions $ for which (20) holds. 
(i) Among all members of @ there exists one that maximizes 


Jof... an. 
(i) A sufficient condition for a member of € to maximize 


[of dn 


is the existence of constants k,,..., k „ such that 


B(x) = d when: f(x) 20d: ifi), 
(21) en 


$(x) = Ov own f(x) if (x). 
i=l 


(iii) If a member of € satisfies (21) with k,,..., km = 0, then it maxi- 
mizes 


fof... du 


*Suggested by Tukey (1949). 
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among all critical functions satisfying 


(22) fofidusc, i=1,...,m. 


(iv) The set M of points in m-dimensional space whose coordinates are 


(fof an..... fet de) 


for some critical function is convex and closed. If (c1,..., Cm) is an inner 
point* of M, then there exist constants k,,..., k,, and a test $ satisfying (20) 
and (21), and a necessary condition for a member of € to maximize 


Jof. de 


is that (21) holds a.e. p. 


Here the term “inner point of M” in statement (iv) can be interpreted as 
meaning a point interior to M relative to m-space or relative to the smallest 
linear space (of dimension < m) containing M. The theorem is correct with 
both interpretations but is stronger with respect to the latter, for which it 
will be proved. 

We also note that exactly analogous results hold for the minimization of 
Sofm+ı dp. 


Proof. (i): Let ($,) be a sequence of functions in € such that fo, fm 41 dp 
tends to sup,/¢f,,,,4#. By the weak compactness theorem for critical 
functions (Theorem 3 of the Appendix), there exists a subsequence ($,) 
and a critical function $ such that 


[9f fof, dn for k=l.. mil. 


It follows that $ is in € and maximizes the integral with respect to fuii di 
within €. 

(ii) and (iii) are proved exactly as was part (ii) of Theorem 1. 

(iv): That M is closed follows again from the weak compactness theorem, 
and its convexity is a consequence of the fact that if $, and $, are critical 
functions, so is ag, + (1 — a)@, for any 0 < a < 1. If N (see Figure 2) is 


*A discussion of the problem when this assumption is not satisfied is given by Dantzig and 
Wald (1951). 
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Figure 2 


the totality of points in (m + 1)-dimensional space with coordinates 


(fet an..... fetu. 


where $ ranges over the class of all critical functions, then N is convex and 
closed by the same argument. Denote the coordinates Of a general point in 
M and N by (u,,..., um) and (u,,..., u,, 41) respectively. The points of N, 
the first m coordinates of which are c,,...,c,,, form a closed interval 
[£*,.c 5^] 

Assume first that c* < c**. Since (c,,..., Cm, c**) is a boundary point 
of N, there exists a hyperplane II through it such that every point of N lies 
below or on II. Let the equation of II be 


m+1 m 
2 ku, = 9» kici + k, ,ac**. 
i=1 i=1 


Since (c,,..., Cm) is an inner point of M, the coefficient k,,41 * 0. To see 
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this, let c* < c < c**, so that (c,...,¢,,»¢) is an inner point of N. Then 
there exists a sphere with this point as center lying entirely in N and hence 
below II. It follows that the point (c;,..., Cm, c) does not lie on II and 
hence that k,,,, # 0. We may therefore take k,,,, = —1 and see that for 
any point of N 


m m 
= ** — 
Umar — dy kiti S Cuin Y kici 
i=l i=1 


That is, all critical functions ó satisfy 


fe... - Eus Jelma- Eua. 
i=l 


i=l 


where $** is the test giving rise to the point (¢),-.-, Cm; c**). Thus $** is 
the critical function that maximizes the left-hand side of this inequality. 
Since the integral in question is maximized by putting $ equal to 1 when 
the integrand is positive and equal to 0 when it is negative, $** satisfies (21) 
a.e. p. 

If c* = c**, let (ci,..., Ch) be any point of M other than (c,,..., Cm) 
We shall show now that there exists exactly one real number c’ such that 


(ct, ..., Ch, c^) is in N. Suppose to the contrary that (c... Cm €) and 
(c1, ..., Ch, C’) are both in N, and consider any point (ef^... , Cm» c") of N 


such that (c,,...,c,,) is an interior point of the line segment joining 
(c1, ...,c^,) and (cf^..., cH). Such a point exists since (c;,..., Cm) is an 
inner point of M. Then the convex set spanned by the three points 
(ct, ..., Clay €^), (£f... Chay €^) and (ets. «s Cm C^) is contained in N and 
contains points (c;,..., Cm» €) and (cy... Cm? C) with c < c, which is a 
contradiction. Since N is convex, contains the origin, and has at most one 
point on any vertical line u; = Cj,---» uy — c/, it is contained in a 
hyperplane, which passes through the origin and is not parallel to the 
u,,,1-axis. It follows that 


m 
Solms du= X k, fof du 
i=1 
for all $. This arises of course only in the trivial case that 
m 
fout kf 4*9 
i=l 


and (21) is satisfied vacuously. 
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Corollary 4. Let p,,..., Pins Pm+1 be probability densities with respect to 
a measure p, and let 0<a<1. Then there exists a test $ such that 
E,o(X) = a (i =1,..., m) and E, ,,6(X) > a, unless p,,, = Xj 1K; pi, 
a.e. p. 


Proof. The proof will be by induction over m. For m = 1 the result 
reduces to Corollary 1. Assume now that it has been proved for any set of m 
distributions, and consider the case of m + 1 densities p,,..., Pm+1- If 
Pi» -- Pm are linearly dependent, the number of p, can be reduced and the 
result follows from the induction hypothesis. Assume therefore that 
Py Pm are linearly independent. Then for each j = 1,..., m there exist 
by the induction hypothesis tests @, and 4j such that E,$( X) = E,4;(X) = 
a for all /21,...,/—1, j * l..., m and E,o(X) < a < E, S (X). It 
follows that the point of m-space for which all m coordinates are equal to a 
is an inner point of M, so that Theorem S(iv) is applicable. The test 
(x) =a is such that Ej$(X) =a for i — 1,..., m. If among all tests 
satisfying the side conditions this one is most powerful, it has to satisfy (21). 
Since 0 « a « 1, this implies 


m 
Pm+1 = yy kp; a.c. p, 
i=] 


as was to be proved. 


The most useful parts of Theorems 1 and 5 are the parts (ii), which give 
sufficient conditions for a critical function to maximize an integral subject 
to certain side conditions. These results can be derived very easily as follows 
by the method of undetermined multipliers. 


Lemma 3. ; Let F,,..., F,,1 be real-valued functions defined over a space 
U, and consider the problem of maximizing F,4,(u) subject to F(u) = 
€; (i=1,...,m). A sufficient condition for a point u Satisfying the side 


conditions to be a solution of the given problem is that among all points of U it 
maximizes 


Failu) = È Fu) 


for some k,,..., k p: 
When applying the lemma one usually carries out the maximization for 


arbitrary k’s, and then determines the constants so as to satisfy the side 
conditions. 
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Proof. If u is any point satisfying the side conditions, then 


E.) ~ ERG) < F9) - È EO), 


i=l 


and hence F,,,;(u) < F,,,;(u°). 

As an application consider the problem treated in Theorem 5. Let U be 
the space of critical functions ¢, and let F,(¢) = Jf, du. Then a sufficient 
condition for $ to maximize F,,,($), subject to F,() = c; is that it 
maximizes F,,,,(¢) — Ek,F(9) = [(fna1 — Ek,f,)ó du. This is achieved 
by setting (x) = 1 or Oas f,,,(x) > or < Ek, f(x). 


7. TWO-SIDED HYPOTHESES 


UMP tests exist not only for one-sided but also for certain two-sided 
hypotheses of the form 


(23) H:0<0,01r0>0, (0,<4,). 


Such testing problems occur when one wishes to determine whether given 
specifications have been met concerning the proportion of an ingredient in a 
drug or some other compound, or whether a measuring instrument, for 
example a scale, is properly balanced. One then sets up the hypothesis that 
6 does not lie within the required limits, so that an error of the first kind 
consists in declaring 6 to be satisfactory when in fact it is not. In practice, 
the decision to accept H will typically be accompanied by a statement of 
whether @ is believed to be <6, or = 4). The implications of H are, 
however, frequently sufficiently important so that acceptance will in any 
case be followed by a more detailed investigation. If a manufacturer tests 
each precision instrument before releasing it and the test indicates an 
instrument to be out of balance, further work will be done to get it properly 
adjusted. If in a scientific investigation the inequalities à < 0, and 0 > 0, 
contradict some assumptions that have been formulated, a more complex 
theory may be needed and further experimentation will be required. In such 
situations there may be only two basic choices, to act as if 0, < 0 < @, or to 
carry out some further investigation, and the formulation of the problem as 
that of testing the hypothesis H may be appropriate. In the present section 
the existence of a UMP test of H will be proved for exponential families. 


Theorem 6. 
G) For testing the hypothesis H: 0 < 0, or 0 = b, (0, < 0) against the 
alternatives K: 0, < 0 < 0, in the one-parameter exponential family (12) 
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there exists a UMP test given by 


1 when C,<T(x)<C, (C, « Cj), 
(24) (x) =(y; when T(x)9- C, i=1,2, 
0 when T(x)< Gor > Q. 


where the C's and y’s are determined by 
(25) E(X) = Eyo(X) = a. 


(ii) This test minimizes Ey@(X) subject to (25) for all 0 < 0, and > 4). 

(iii) For 0 < a < 1 the power function of this test has a maximum at a 
point 0 between 0, and 0; and decreases strictly as 0 tends away from bù in 
either direction, unless there exist two values t,, t, such that P,(T(X) = 1} 
+ P,(T(X) = t,} =1 for all 0. 


Proof. (i): One can restrict attention to the sufficient statistic T = T( X), 
the distribution of which by Lemma 8 of Chapter 2 is 


dP,(t) = C(@) e2 dv(t), 


where Q(0) is assumed to be strictly increasing. Let 6, < 0' < 0,, and 
consider first the problem of maximizing E,.W(T) subject to (25) with 
$(x) = V[T(x)]. If M denotes the set of all points (Eg (T), Ej Jj (T)) as y 
ranges over the totality of critical functions, then the point (a, a) is an inner 
point of M. This follows from the fact that by Corollary 1 the set M 
contains points (a, u;) and (a, u;) with u, < a < u, and that it contains all 
points (u, u) with 0 « u « 1. Hence by part (iv) of Theorem 5 there exist 


constants k,, k, and a test W(t) such that (x) = V,[Ti satisfies (25) 
and that y(t) = 1 when $ ae aia 


K,C(6,) e99' + k ,C(0,) e20! < c(gr) eo^ 


and therefore when 


aye + ae’! <1 (b, <0 < b,), 


and Yo(t) = 0 when the left-hand side is > 1. Here the a’s cannot both be 
< 0, since then the test would always reject. If one of the a’s is < 0 and 
the other one is > 0, then the left-hand side is strictly monotone, and the 
test is of the one-sided type considered in Corollary 2, which has a strictly 
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monotone power function and hence cannot satisfy (25). Since therefore 
both a’s are positive, the test satisfies (24). It follows from Lemma 4 below 
that the C’s and y’s are uniquely determined by (24) and (25), and hence 
from Theorem 5(iii) that the test is UMP subject to the weaker restriction 
E,¥(T) < a (i= 1,2). To complete the proof that this test is UMP for 
testing H, it is necessary to show that it satisfies E,y(T) < a for 0 < 6, 
and @ > 6. This follows from (ii) by comparison with the test p(t) = a. 

(ii): Let 0^ < 6,, and apply Theorem 5(iv) to minimize Ej,$( X) subject 
to (25). Dividing through by e?(^?', the desired test is seen to have a 
rejection region of the form 


ae +a,e%<1 (b; «0« bj). 


Thus it coincides with the test Y(t) obtained in (i). By Theorem 5(iv) the 
first and third conditions of (24) are also necessary, and the optimum test is 
therefore unique provided P(T = C;) = 0. 

(iii); Without loss of generality let Q(0) = 0. It follows from (i) and the 
continuity of B(@) = E(X) that either B(@) satisfies (iii) or there exist 
three points 0^ < 0" < 8" such that &(0") < B(0") = B(0'") = c, say. 
Then 0 < c < 1, since B(6’) = 0 (or 1) implies $(t) = 0 (or 1) ae. » and 
this is excluded by (25). As is seen by the proof of (i), the test maximizes 
E,,9( X) subject to Ej9(X) = Eg» (X) = c for all 0’ < 0" < 6". How- 
ever, unless T takes on at most two values with probability 1 or all 0, 
Po» Por, Py are linearly independent, which by Corollary 4 implies B(0") 
AD 

In order to determine the C's and y's, one will in practice start with 
some trial values Cf, yř, find C£, yf such that B*(0,) = a, and compute 
B*(6,), which will usually be either too large or too small. For the selection 
of the next trial values it is then helpful to note that if B*(0,) < a, the 
correct acceptance region is to the right of the one chosen, that is, it satisfies 
either C, > Ct or C, = Cf and y, < yi, and that the converse holds if 
B*(0,) > a. This is a consequence of the following lemma. 


Lemma 4. Let p,(x) satisfy the assumptions of Lemma 2v). 

G) If $ and 4* are two tests satisfying (24) and E,o(T) = E,¢*(T), 
and if $* is to the right of , then B(@) < or > B*(8) as 8 » 0, or < 6,. 

Gi) If @ and ¢* satisfy (24) and (25), then ọ = $* with probability one. 

Proof. (i): The result follows from Lemma 2v) with y = $* — 9. 

(ii): Since E,9(T) = Eo*(T), 9* lies either to the left or the right of 9, 
and application of (i) completes the proof. 


104 UNIFORMLY MOST POWERFUL TESTS [3.8 


Although a UMP test exists for testing that 0 < 0, or >, in an 
exponential family, the same is not true for the dual hypothesis H: 
0, € 0 « 0, or for testing 0 = 6, (Problem 31). There do, however, exist 
UMP unbiased tests of these hypotheses, as will be shown in Chapter 4. 


8. LEAST FAVORABLE DISTRIBUTIONS 


It is a consequence of Theorem 1 that there always exists a most powerful 
test for testing a simple hypothesis against a simple alternative. More 
generally, consider the case of a Euclidean sample space; probability 
densities fj, @ € w, and g with respect to a measure p; and the problem of 
testing H : fj, Ó € c, against the simple alternative K : g. The existence of a 
most powerful level-a test then follows from the weak compactness theorem 
for critical functions (Theorem 3 of the Appendix) as in Theorem 5(i). 

Theorem 1 also provides an explicit construction for the most powerful 
test in the case of a simple hypothesis. We shall now extend this theorem to 
composite hypotheses in the direction of Theorem 5 by the method of 
undetermined multipliers. However, in the process of extension the result 
becomes much less explicit. Essentially it leaves open the determination of 
the multipliers, which now take the form of an arbitrary distribution. In 
specific problems this usually still involves considerable difficulty. 

From another point of view the method of attack, as throughout the 
theory of hypothesis testing, is to reduce the composite hypothesis to a 
simple one. This is achieved by considering weighted averages of the 
distributions of H. The composite hypothesis H is replaced by the simple 
hypothesis H, that the probability density of X is given by 


a(x) = f fo(x) a (0). 


where A is a probability distribution over w. The problem of finding a 
suitable A is frequently made easier by the following consideration. Since H 
provides no information concerning 6 and since H 4 is to be equivalent to H 
for the purpose of testing against g, knowledge of the distribution A should 
provide as little help for this task as possible. To make this precise suppose 
that @ is known to have a distribution A. Then the maximum power Ba that 
can be attained against g is that of the most powerful test ó, for testing Hy, 
against g. The distribution A is said to be least favorable (at level a) if for 
all A’ the inequality B, < By holds. 


Theorem 7. Let a o-field be defined over w such that the densities f(x) 
are jointly measurable in 8 and x. Suppose that over this o-field there exists a 
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probability distribution A such that the most powerful level-a test $, for 
testing H, against g is of size < a also with respect to the original hypothesis 


(i) The test p, is most powerful for testing H against g. 

(ii) If $4 is the unique most powerful level-a test for testing Hy against g, 
it is also the unique most powerful test of H against g. 

(iii) The distribution A is least favorable. 


Proof. We note first that A, is again a density with respect to p, since 
by Fubini’s theorem (Theorem 3 of Chapter 2) 


[8460 du(x) = f ah (0 fhal) du) = [ 4A(9) = 1. 


Suppose that $, is a level-a test for testing H, and let $* be any other 
level-a test. Then since Ej9*( X) < a for all 6 € w, we have 


Jhal) du(x) = [Evi CX dA (0) < a. 


Therefore $* is a level-a test also for testing H, and its power cannot 
exceed that of $4. This proves (i) and (ii). If A’ is any distribution, it follows 
further that $, is a level-a test also for testing Hy, and hence that its power 
against g cannot exceed that of the most powerful test, which by definition 
is By. 

The conditions of this theorem can be given a somewhat different form 
by noting that $, can satisfy [,,E,,(X) dA(0) = a and Eyg,(X) < a for 
all 0 € w only if the set of "s with Ey@,(X) = a has A-measure one. 


Corollary 5. Suppose that A is a probability distribution over w and that 
w! is a subset of w with A(w’) = 1. Let $y be a test such that 


1 if g(x) > k f(x) 4A (0), 


(26) d(x) = 
0 if g(x) < kf f(x) 4 (0). 


Then $, is a most powerful level-a test for testing H against g provided 


(27) Epp (X) = sup Esp (X) =a for F Ew. 
bew 


106 UNIFORMLY MOST POWERFUL TESTS [3.8 


Theorems 2 and 6 constitute two simple applications of Theorem 7. The 
set o over which the least favorable distribution A is concentrated consists 
of the single point 6, in the first of these examples and of the two points 6, 
and 6, in the second. This is what one might expect, since in both cases 
these are the distributions of H that appear to be "closest" to K. Another 
example in which the least favorable distribution is concentrated at a single 
point is the following. 


Example 8. Sign test. The quality of items produced by a manufacturing 
process is measured by a characteristic X such as the tensile strength of a piece of 
material, or the length of life or brightness of a light bulb. For an item to be 
satisfactory X must exceed a given constant u, and one wishes to test the hypothesis 
H: p = po, where 


p=P{X<u} 


is the probability of an item being defective. Let X,,..., X, be the measurements of 
n sample items, so that the X’s are independently distributed with common 
distribution about which no knowledge is assumed. Any distribution on the real line 
can be characterized by the probability p together with the conditional probability 
distributions P. and P, of X given X<u and X» u respectively. If the 
distributions P_ and P, have probability densities p_ and p,, for example with 
respect to p = P_+ P,, then the joint density of X,,..., X, at a sample point 
Xj... , X, Satisfying 

Xs XQ SUS Xy X 


"r nm 


is 
p'(1— p) "p-Qxu)** px, ) pe (a) 7" pa (xj,...)- 


Consider now a fixed alternative to H, sa i 
, Say (py, P_, P,), with < po. One 
would then expect the least favorable distribution A over H "to Mit probability 1 


to the distribution (py, P , P,) since this appears to be close 
d u (Po P_, t to the selected 
alternative. With this choice of A, the test (26) becoihes Mac 


m n-m 
g(x) =lor0 as (2) (5) SOC e C, 
Po 4o 


and hence as m < or > C. The test therefore rejects when the number M of 


defectives is sufficiently small, or more pecisel i ili 
P tet Me pecisely, when M « C and with probability 


(28) P(M«C)*yP(M-C)-a for p=p. 


The distribution of M is the binomial distribution b( 
P, n), and does not depend on 
P, and P_. As a consequence, the power function of the test depends Eds on p 
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and is a decreasing function of p, so that under H it takes on its maximum for 
p = po. This proves A to be least favorable and $, to be most powerful. Since the 
test is independent of the particular alternative chosen, it is UMP. 

Expressed in terms of the variables Z; = X, — u, the test statistic M is the 
number of variables < 0, and the test is the so-called sign test (cf. Chapter 4, 
Section 9). It is an example of a nonparametric test, since it is derived without 
assuming a given functional form for the distribution of the X"s such as the normal, 
uniform, or Poisson, in which only certain parameters are unknown. 

The above argument applies, with only the obvious modifications, to the case 
that an item is satisfactory if X lies within certain limits: u « X « v. This occurs, 
for example, if. X is the length of a metal part or the proportion of an ingredient in 
a chemical compound, for which certain tolerances have been specified. More 
generally the argument applies also to the situation in which X is vector-valued. 
Suppose that an item is satisfactory only when X lies in a certain set S, for 
example, if all the dimensions of a metal part or the proportions of several 
ingredients lie within specified limits. The probability of a defective is then 


p-P(Xe$), 


and P. and P, denote the conditional distributions of X given XeSand XES 
respectively. As before, there exists a UMP test of H : p 2 po, and it rejects H when 
the number M of defectives is sufficiently small, with the boundary of the test being 
determined by (28). 


A distribution A satisfying the conditions of Theorem 7 exists in most of 
the usual statistical problems, and in particular under the following assump- 
tions. Let the sample space be Euclidean, let w be a closed Borel set in 
s-dimensional Euclidean space, and suppose that f(x) is a continuous 
function of 0 for almost all x. Then given any g there exists a distribution 
A satisfying the conditions of Theorem 7 provided 


lim. ff, (x) du(x) = 0 
n-o "Ss 
for every bounded set S in the sample space and for every sequence of 
vectors 0, whose distance from the origin tends to infinity. 

From this it follows, as did Corollaries 1 and 4 from Theorems 1 and 5, 
that if the above conditions hold and if 0 « a « 1, there exists a test of 
power B > a for testing H : fy, 9 € w, against g unless g — UD dA(@) for 
some A. An example of the latter possibility is obtained by letting fo, and g 
be the normal densities N(6, o2) and N(0, o7) respectively with og < oj. 
(See the following section.) 

The above and related results concerning the existence and structure of 
least favorable distributions are given in Lehmann (1952) (with the require- 
ment that w be closed mistakenly omitted), in Reinhardt (1961), and in 
Krafft and Witting (1967), where the relation to linear programming 1s 
explored. 
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9. TESTING THE MEAN AND VARIANCE OF A 
NORMAL DISTRIBUTION 


Because of their wide applicability, the problems of testing the mean £ and 
variance o? of a normal distribution are of particular importance. Here and 
in similar problems later, the parameter not being tested is assumed to be 
unknown, but will not be shown explicitly in a statement of the hypothesis. 
We shall write, for example, o < o, instead of the more complete statement 
o X0, — œ% <$ < oo. The standard (likelihood-ratio) tests of the two 
hypotheses ø < aj and  < £, are given by the rejection regions 


(29) X(x-xy2c 

and 

(30) LC IRSE S 
med 25x; zi xy 


The corresponding tests for the hypotheses o > 9, and £ > £, are obtained 
from the rejection regions (29) and (30) by reversing the inequalities. As will 
be shown in later chapters, these four tests are UMP both within the class of 
unbiased and within the class of invariant tests (but see Chapter 5, Section 4 
for problems arising when the assumption of normality does not hold 
exactly). However, at the usual significance levels only the first of them is 
actually UMP. 

Let X,..., X, be a sample from N(£,o?), and consider first the 
hypotheses H; : > og and H;: 0 < oy, anda simple alternative K : £ = £i, 
a = o. It seems reasonable to suppose that the least favorable distribu- 
tion A in the (£, )-plane is concentrated on the line o = go. Since Y = 
LX,/n = X and U = X( X, — X)? are sufficient statistics for the parameters 


($, o), attention can be restricted to these variables. Their joint density 
under H, is 


u 


saz) fox |- e J dA(é), 


Cou" exp (- 


while under K it is 


(n-3)/ A n 
is (za - Z3 e] 
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The choice of A is seen to affect only the distribution of Y. A least 
favorable A should therefore have the property that the density of Y under 
His 


f avo a 4A(£), 


comes as close as possible to the alternative density, 


Yn n i 
Wn 3 252 U - &) | 


At this point one must distinguish between H, and H,. In the first case 
9, < o. By suitable choice of A the mean of Y can be made equal to £j, but 
the variance will if anything. be increased over its initial value oj. This 
suggests that the least favorable distribution assigns probability 1 to the 
point £ = £, since in this way the distribution of Y is normal both under H 
and K with the same mean in both cases and the smallest possible 
difference between the variances. The situation is somewhat different for H;, 
for which a, < o;. If the least favorable distribution A has a density, say A’, 
the density of Y under H, becomes 


ia oe j aiU = evo d£. 


This is the probability density of the sum of two independent raridom 
variables, one distributed as N(0, a2/n) and the other with density N(£). If 
A is taken to be N(£, (o2 — og)/n), the distribution of Y under Hy 
becomes N(£,, o2/n), the same as under K. 

We now apply Corollary 5 with the distributions A suggested above. For 
H, it is more convenient to work with the original variables than with Y and 


U. Substitution in (26) gives ¢(x) = 1 when 


(270) "exp |- a Mo- a 


————————- » €, 
7 1 
(2702) "exp E p XG,- a 
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that is, when 
(31) 2 (x; = ay s C. 


To justify the choice of A, one must show that 
PECX- &) < cle, o) 


takes on its maximum over the half plane o > o at the point £ = £j, 
o = o. For any fixed ø, the above is the probability of the sample point 
falling in a sphere of fixed radius, computed under the assumption that the 
X’s are independently distributed as N(£, 07). This probability is maxi- 
mized when the center of the sphere coincides with that of the distribution, 
that is, when ¢ = ,. (This follows for example from Problem 25 of Chapter 
7.) The probability then becomes 


P[z(&-5]. € 


where V,,..., V, are independently distributed as N(0, 1). This is a decreas- 
ing function of o and therefore takes on its maximum when o = Op- 

In the case of H,, application of Corollary 5 to the sufficient statistics 
(Y,U) gives $(y, u) = 1 when 


5 = P{ Ev? < 5}, 


Pal u n 
Cut "e ( - zie |- 25 U E a 


xd u 
EINER fev|-z56 F 2 N(ë) ae 


1 1 
= ee |- (5 - =| 20, 
2\ of og 


(32) u-Y(x-x)zc. 


that is, when 


Since the distribution of D(X, — Y)?/s? does not d 

he of epend on £ or a, the 
probability P(X(X,—- X yz C|& c) is independent of é and increases 
with c, so that the conditions of Corollary 5 are satisfied. The test (32), 
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being independent of £, and o,, is UMP for testing o < op against a > op. 
It is also seen to coincide with the likelihood-ratio test (29). On the other 
hand, the most powerful test (31) for testing o > oy against o < øg does 
depend on the value £, of under the alternative. 

It has been tacitly assumed so far that n > 1. If n — 1, the argument 
applies without change with respect to Hj, leading to (31) with n = 1. 
However, in the discussion of H, the statistic U now drops out, and Y 
coincides with the single observation X. Using the same A as before, one 
sees that X has the same distribution under H, as under K, and the test $, 
therefore becomes $,(x) = a. This satisfies the conditions of Corollary 5 
and is therefore the most powerful test for the given problem. It follows that 
a single observation is of no value for testing the hypothesis H,, as seems 
intuitively obvious, but that it could be used to test H, if the class of 
alternatives were sufficiently restricted. 

The corresponding derivation for the hypothesis £ < £5 is less straight- 
forward. It turns out* that Student's test given by (30) is most powerful if 
the level of significance a is > }, regardless of the alternative £ > £o, 04. 
This test is therefore UMP for a > 4. On the other hand, when a < } the 
most powerful test of H rejects when L(x; — a)? € b, where the constants a 
and b depend on the alternative (£,, 01) and on a. Thus for the significance 
levels that are of interest, a UMP test of H does not exist. No new problem 
arises for the hypothesis £ > £o, since this reduces to the case just consid- 
ered through the transformation Y, = £y — (X; — £o). 


10. PROBLEMS 


Section 2 
1. Let X,,..., X, be a sample from the normal distribution N(£, o°). 


(i) If o = o (known), there exists a UMP test for testing H : £ < £y against 
£ > £o, which rejects when E(X, — £o) is too large. ) 
Gi) If ¢ = £, (known), there exists a UMP test for testing H: ¢ < % against 
K: o > o, which rejects when E(X, — £o)? is too large. 
2. UMP test for U(0,0). Let X= (Xy,..., X,) be a sample from the uniform 
distribution on (0, 6). 
i i i ; i for 
(i) For testing H: 6 < 6 against K: 6 > 6, any test is UMP at level a 
which Ej$(X) = a, E(X) «a for 0< %, and $(x)=1 when 
max(x;,..., Xn) > bo: à 
Gi) For testing H:6 = 0 against K: 0 # & a unique UMP test exists, and 
is given by é(x) = 1 when max(x1,.--»%n) > % or max(;,-.., Xn) S o 
Ya, and $(x) = 0 otherwise. 


*See Lehmann and Stein (1948). 
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[G): For each @> 6, determine the ordering established by r(x) = 
Po(x)/Po,(x) and use the fact that many points are equivalent under this 
ordering. 

(ii): Determine the UMP tests for testing 0 = 9 against 0 < @ and combine 
this result with that of part (i).] 


UMP test for exponential densities. Let X,,...,X, be a sample from the 
exponential distribution E(a, b) of Chapter 1, Problem 18, and let X, = 
min(X,,..., Xp): 


(i) Determine the UMP test for testing H: a = aj against K: a * ay when 
b is assumed known. 

(i) The power of any MP level-a test of H: a = ay against K: a = a, < ag 
is given by 


B*(a,) *1— (1— a)e^"t2-aY^, 


(ii) For the problem of part (i), when b is unknown, the power of any level a 
test which rejects when 


Xa) — 40 


[x capa] 455 20 


against any alternative (a, b) with a, < a is equal to 8*(a,) of part (ii) 
(independent of the particular choice of C, and C;). 

(v) The test of part (iii) is a UMP level-a test of H:a = ag against 
K: a+ a, (b unknown). 

(v) Determine the UMP test for testing H:a — ay, b= by against the 
alternatives a < ag, b < by. 

(vi) Explain the (very unusual) existence in this case of a UMP test in the 


presence of a nuisance parameter [part (iv)] and for a hypothesis specify- 
ing two parameters [part (v)]. 


[G): the variables Y, = e-*/^ are a sample from the uniform distribution on 
(0, e7*/^)] 


Note. For more general versions of parts (ii)-(iv) see Takeuchi (1969) and 
Kabe and Laurent (1981). 


The following example Shows that the power of a test can sometimes be 
increased by selecting a random rather than a fixed sample size even when the 
randomization does not depend on the observations. Let X,,..., X, be inde- 


pendently distributed as N(0, 1), and consider the problem of testing H: 0 = 0 
against K: 0 = 0, > 0. 


(i) The power of the most powerful test as a function of the sample size n is 
not necessarily concave. 
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(i) In particular for a = .005, 0, = 1, better power is obtained by taking 2 
or 16 observations with probability } each than by taking a fixed sample 
of 9 observations. 

(ii) The power can be increased further if the test is permitted to have 
different significance levels a, and a, for the two sample sizes and it is 
required only that the expected significance level be equal to a — .005. 
Examples are: (a) with probability } take m, — 2 observations and 
perform the test of significance at level a, = .001, or take n, = 16 
observations and perform the test at level a, = .009; (b) with probability 
+ take n, = 0 or n; = 18 observations and let the respective significance 
levels be a, = 0, a; = .01. 

Note. This and related examples were discussed by Kruskal in a semi- 
nar held at Columbia University in 1954. A more detailed investigation 
of the phenomenon has been undertaken by Cohen (1958). 


5. If the sample space Z is Euclidean and Pp, P, have densities with respect to 
Lebesgue measure, there exists a nonrandomized most powerful test for testing 
P, against P, at every significance level a.* 
[This is a consequence of Theorem 1 and the following lemma.‘ Let f > 0 and 
[Af (x) dx = a. Given any 0 < b < a, there exists a subset B of A such that 


Jaf(x) dx = b] 


6. Fully informative statistics. A statistic T is fully informative if for every 

decision problem the decision procedures based only on T form an essentially 
complete class. If P is dominated and T is fully informative, then T is 
sufficient. 
[Consider any pair of distributions Py, P, € P with densities po, p, and let 
E; = pj/(po + pi). Suppose that T is fully informative, and let » be the 
subfield induced by T. Then »% contains the subfield induced by (go, 81) 
since it contains every rejection region which is unique most powerful for 
testing P, against P, (or P, against Py) at some level a. Therefore, T is 
sufficient for every pair of distributions (Py, P,), and hence by Problem 10 of 
Chapter 2 it is sufficient for 2.] 


Section 3 
7. Let X be the number of successes in n independent trials with probability: p 


of success, and let (x) be the UMP test (9) for testing p < po against p > po 
at level of significance a. 


(i) For n — 6, p = 25 and the levels a = .05,.1,.2 determine C and y, 
and find the power of the test against p, = 3, 4, 5, .6,.7. 


"For more general results concerning the possibility of dispensing with randomized 
Procedures, see Dvoretzky, Wald, and Wolfowitz (1951). ; ' 

}For a proof of this lemma see Halmos (1974, p. 174.) The lemma is a special case of a 
theorem of Lyapounov (see Blackwell (1951a).) 
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10. 


11. 


(ii) If py =.2 and a = .05, and it is desired to have power £ > .9 against 
pi = 4, determine the necessary sample size (a) by using tables of the 
binomial distribution, (b) by using the normal approximation.* 

(ii) Use the normal approximation to determine the sample size required 
when a = .05, B = 9, p, = 01, p, = .02. 


(i) A necessary and sufficient condition for densities pọ(x) to have mono- 
tone likelihood ratio in x, if the mixed second derivative 
glog ps (x)/00 Ax exists, is that this derivative is > 0 for all @ and x. 

(i) An equivalent condition is that 


(x) 8^ p(x) > 9 po( x) ap (x) 
POX) a9 ax 80 ðx 


Let the probability density p; of X have monotone likelihood ratio in T(x), 
and consider the problem of testing H : 0 < 6, against 0 > 6. If the distribu- 
tion of T is continuous, the p-value à of the UMP test is given by â = P4(T 
21), where ¢ is the observed value of T. This holds also without the 
assumption of continuity if for randomized tests à is defined as the smallest 
significance level at which the hypothesis is rejected with probability 1. 


Let X,,..., X, be independently distributed with density (20) !e- */?9, x > 0, 
and let Y, < --- < Y, be the ordered X's. Assume that Y, becomes available 
first, then Y,, and so on, and that observation is continued until Y, has been 
observed. On the basis of ¥,,..., Y, it is desired to test H : 0 6, = 1000 at 
level a = .05 against 0 < 6. 


for all ô and x. 


(i) Determine the rejection region when r = 4, and find the power of the test 
against 0, = 500. 


(ii) Find the value of r required to get power B > .95 against this alternative. 


[In Problem 14, Chapter 2, the distribution of [D7_,Y, + (n — r)Y,]/0. was 
found to be x? with 2r degrees of freedom. 


When a Poisson process with rate À is observed for a time interval of length 7, 
the number X of events occurring has the Poisson distribution P(Ar). Under 
an alternative Scheme, the process is observed until r events have occurred, and 
the time T of observation is then a random variable such that 2AT has a 
X^-distribution with 2r degrees of freedom. For testing H:À < A, at level a 


one can, under either design, obtain a specified power £ against an alternative 
A, by choosing 7 and r sufficiently large. 


(i The ratio of the time of observation required for this purpose under the 
first design to the expected time required under the second is Ar/r. 

(ii) Determine for which values of A each of the two designs is preferable 

when Ay = 1, A, = 2, a = .05, B= 9. 


*Tables and approximations are discussed, for example, in Chapter 3 of Johnson and Kotz 


(1969). 
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12. 


13. 


14. 


Let X =(Xj,..., X,) be a sample from the uniform distribution U(0,8 1). 


(i) For testing H: @ < 6 against K:0 > 6, at level a there exists a UMP 
test which rejects when min(X;,,..., X,) > % + C(a) or 
max( X,,..., X,) > 6, + 1 for suitable C(a). 

(i) The family U(0, 0 + 1) does not have monotone likelihood ratio. [Ad- 
ditional results for this family are given in Birnbaum (1954) and Pratt 
(1958).] 


[Gi) By Theorem 2, monotone likelihood ratio implies that the family of UMP 
tests of H: 0 < 0, against K:0 > generated as a varies from 0 to 1 is 
independent of 6, ]. 


Let X be a single observation from the Cauchy density given at the end of 
Section 3. 


(i) Show that no UMP test exists for testing 0 = 0 against 0 > 0. 

(i) Determine the totality of different shapes the MP level-a rejection region 
for testing 0 = 6, against @ = 0, can take on for varying a and 0, — b. 

Extension of Lemma 2. Let Py and P, be two distributions with densities 

Po» pı such that p,(x)/po(x) is a nondecreasing function of a real-valued 

statistic T(x). 


(i) If T has probability density p; when the original distribution is P,, then 
pi (t)/p (t) is nondecreasing in t. 
(ii) Egy(T) € EjJ(T) for any nondecreasing function y. 
(iii) If p,(x)/po(x) is a strictly increasing function of t= T(x), so is 
pi(t)/po(t), and Eoy(T) < Eyy(T) unless ¥[7(x)] is constant a.e. 
(Py + P,) or Egy(T) = Eyy(T) = £0. 
(iv) For any distinct distributions with densities po, Pi» 


p(X) p(X) 
-o < mal CO] «| Gy | 5 


[@: Without loss of generality suppose that p,(x)/po(x) = T(x). Then for 
any integrable $, 


[eG pet) dvo = fol rGO)rG) m2 dux) = JANO dC), 


and hence pj(1t)/po(t) = t a.e. ) r 
Gv): The possibility Eolog[ pı(X)/Po(X)] = © 18 excluded, since by the 
convexity of the function log, 


X (X) 
seg] sh [n0 * 
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Similarly for E,. The strict inequality now follows from (iii) with T(x) = 
Pi(X)/Po(*)-] 


If F, F, are two cumulative distribution functions on the real line, then 
F,(x) € F(x) for all x if and only if Epy(X) < Ej ( X) for any nondecreas- 
ing function y. 


Section 4 
If the experiment ( f, g) is more informative than ( f’, g^), then (g, f) is more 
informative than (g’, f’). 


Conditions for comparability. 


() Let X and X' be two random variables taking on the values 1 and 0, and 
suppose that P(X = 1) = po, P(X' = 1} = pj or that P(X = 1} = pi, 
P(X' = 1) = pj. Without loss of generality let py < pj, po < Pi» Po < Pi- 
(This can be achieved by exchanging X with X' and by exchanging the 
values 0 and 1 of one or both of the variables) Then X is more 
informative than X' if and only if (1 — p,)(1 — p) € (1 — po)(1 — Ppi). 

(ii) Let W,U, be independently uniformly distributed over (0,1), and let 
Y=1 if X-1 and U, « y, and if X 2 0 and U) < Yọ and Y - 0 
otherwise. Under the assumptions of (i) there exist 0 < Yọ, y, < 1 such 

that P(Y = 1} = p; when P( X = 1} = p, (i = 0,1) provided (1 — p,)(1 

— po) < (1 — po — pi). This inequality, which is therefore sufficient 

for a sample X,,..., X, from X to be more informative than a sample 

X{,..., Xp from X', is also necessary. Similarly, the condition pop; < 

Popi is necessary and sufficient for a sample from X’ to be more 

informative than one from X. 


[@: The power B(a) of the most powerful level-a test of pọ against p, based 
on X is ap,/po if a < py, and p,  q,45 (a — po) if py < a. One obtains 
the desired result by comparing the graphs of B(a) and B’(a). 

(ii): The last part of (ii) follows from a comparison of the power B,(«) and 
B; (a) of the most powerful level a tests based on X. X, and X. X/ for a close to 
1. The dual condition is obtained from Problem 16.] 


For the z X 2 table described in Example 4, and under the assumption 
px s } made there, a sample from B is more informative than one from A. 
On the other hand, samples from B and B are not comparable. 


[A necessary and sufficient condition for comparability is given in the preced- 
ing problem.] 


In the experiment discussed in Example 5, n binomial trials with probability of 
success p = 1 — e "' are performed for the purpose of testing A = A, against 


A =),. Experiments corresponding to two different values of v are not 
comparable. 
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Section 5 
(i) For n — 5, 10 and 1 — a — .95, graph the upper confidence limits p and 
p* of Example 7 as functions of t = x + u. 


(ii) For the same values of n and a, = a, = .05, graph the lower and upper 
confidence limits p and p. 


Confidence bounds with minimum risk. Let L(0, 0) be nonnegative and nonin- 
creasing in its second argument for 0 < 0, and equal to 0 for 0 > 0. If 0 and 
8* are two lower confidence bounds for @ such that 


P,(0x0') < P,(0* x0") forall 8’ <9, 
then 
E,L(0,0) < E,L(0,0*). 


[Define two cumulative distribution functions F and F* by F(u) = Py{8 < 
u}/P,{0* <9}, F*(u) = P,(0* < u)/P,(0* <9} for u «60, and F(u) = 
F*(u) =1 for u 2 0. Then F(u) < F*(u) for all u, and it follows from 
Problem 15 that 


E,[L(0,8)] = A (8* < 0) [L(0, u) dF(u) 
< b (6* <0} fL(0, u) dF*(u) = E[L(0,0*)].] 


Section 6 


If B(8) denotes the power function of the UMP test of Corollary 2, and if the 
function Q of (12) is differentiable, then 8'(0) > O for all 0 for which 
Q'(0) > 0. 

[To show that B’(8) > 0, consider the problem of maximizing, subject to 
Ej$(X) = a, the derivative B'(05) or equivalently the quantity 
Ej [TOO $ 0M 

Optimum selection procedures. On each member of a population n measure- 
ments (X,,..., X,) = X are taken, for example the scores of n aptitude tests 
which are administered to judge the qualifications of candidates for a certain 
training program. A future measurement Y such as the score in a final test at 
the end of the program is of interest but unavailable. The joint distribution of 
X and Y is assumed known. 


(i) One wishes to select a given proportion a of the candidates in such a way 
as to maximize the expectation of Y for the selected group. This is 
achieved by selecting the candidates for which E(Y|x) = C, where C is 
determined by the condition that the probability of a member being 


118 


24. 


25: 


26. 


27. 


UNIFORMLY MOST POWERFUL TESTS [3.10 
selected is a. When E(Y|x) = C, it may be necessary to randomize in 
order to get the exact value a. 

(i) If instead the problem is to maximize the probability with which in the 
selected population Y is greater than or equal to some preassigned score 
Yo, one selects the candidates for which the conditional probability 
P(Y > yo|x} is sufficiently large. 


[(): Let p(x) denote the probability with which a candidate with measure- 
ments x is to be selected. Then the problem is that of maximizing 


ROLO 4| p*(x) dx 


subject to 
Jo) dx =a] 


The following example shows that Corollary 4 does not extend to a countably 
infinite family of distributions. Let p, be the uniform probability density on [0, 
1 + 1/n], and p, the uniform density on (0, 1). 


(i) Then p, is linearly independent of ( p;, p», ...), that is, there do not exist 
constants c, c;,... such that po = Le, Pa- 


(ii) There does not exist a test @ such that fóp, — a for n — 1,2,... but 


[opo > a. 
Let F,,..., F,,,, be real-valued functions defined over a space U. A sufficient 
condition for ug to maximize F,,,, subject to E(u) < c; (i= 1,..., m) is that 


it satisfies these side conditions, that it maximizes F,,, ,(u) — Lk, F(u) for 


xi Raab k, 20, and that E(ug) — c; for those values i for which 
; ^ 9. 


Section 7 


For a random variable X with binomial distribution b( p, n), determine the 
constants C;, y, (i = 1,2) in the UMP test (24) for testing H: p < 20r € -1 
Mae = .1 and n = 15. Find the power of the test against the alternative 
p=A. 


Totally positive families. A family of distributions with probability densities 
po(x), 0 and x real-valued and varying over € and Z respectively, is said to 
be totally positive of order r (TP,) if for all x, < --. < x, and 0, < =- < 6, 


pam) ce Po,( Xn) 


33) A, = 
(33) m ew chr Sse ea 20 forall n=1,2,...,r. 


3.10] PROBLEMS 119 


28. 


29. 


It is said to be strictly totally positive of order r (STP,) if strict inequality 
holds in (33). The family is said to be (strictly) totally positive of order infinity 
if (33) holds for all n = 1,2,... . These definitions apply not only to probabil- 
ity densities but to any real-valued functions p,(x) of two real variables. 


(i) For r = 1, (33) states that py(x) = 0; for r = 2, that p(x) has mono- 
tone likelihood ratio in x. 
(ii) If a(8) > 0, b(x) > 0, and p(x) is STP,, then so is a(8)b(x) po(x). 
(iii) If a and b are real-valued functions mapping € and Z onto Q’ and 2’ 
and are strictly monotone in the same direction, and if p(x) is (S)TP,, 
then py-(x’) with 0’ = a~1(@) and x’ = b^ (x) is (S)TB, over (Q', 2”). 


Exponential families. The exponential family (12) with T(x) = x and Q(0) 
= 0 is STP,,, with Q the natural parameter space and g= (— 00,00). 

[That the determinant |e**|, i, j — 1,..., n, is positive can be proved by 
induction. Divide the ith column by e^"*, ji — 1,...,n; subtract in the 
resulting determinant the (n — 1)st column from the nth, the (n — 2)nd from 
the (n — 1)st,..., the 1st from the 2nd; and expand the determinant obtained 
in this way by the first row. Then A, is seen to have the same sign as 


M, =en — erma], i,j-22,..n, 


where n; = 6, — 6,. If this determinant is expanded by the first column one 
obtains a sum of the form 


a,(e%*2 — emn) + +++ +a,( eM? — emm) = h(x) — h(x) 
= (x, 7 x1)h'(»2), 


where x, < y; < x,. Rewriting h'(y;) as a determinant of which all columns 
but the first coincide with those of A, and proceeding in the same manner 
with the other columns, one reduces the determinant to |e", i, j = 2,..., n, 
which is positive by the induction hypothesis.] $ 

STP,. Let Ó and x be real-valued, and suppose that the probability densities 
Po(x) are such that pp-(x)/pe(x) is strictly increasing in x for 0 < 0. Then 
the following two conditions are equivalent: (a) For 6, < b, < 0, and ki, k2, 
k, > 0, let 


g(x) = kı Pa (x) — ka pe (x) + ks po (x) 


If g(x) = g(x3) = 0, then the function g is positive outside the interval 
(3, x3) and negative inside. (b) The determinant A, given by (33) is positive 
for all 0, < 6, < 6, xy < x; < xs. [It follows from (a) that the equation 
g(x) = 0 has at most two solutions.] 
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[That (b) implies (a) can be seen for x, < x, < x, by considering the determi- 
nant 


gx) g(x) g(x) 
Po. xj) Po,( x3) Pe x3)|. 
po) Ps xj) Po;( x) 


Suppose conversely that (a) holds. Monotonicity of the likelihood ratios 
implies that the rank of A, is at least two, so that there exist constants 
ky, kz, ky such that g(x) = g(x) = 0. That the k’s are positive follows 
again from the monotonicity of the likelihood ratios.] 


Extension of Theorem 6. The conclusions of Theorem 6 remain valid if the 
density of a sufficient statistic T (which without loss of generality will be taken 
to be X), say ps(x), is STP, and is continuous in x for each 0. 

[The two properties of exponential families that are used in the proof of 
Theorem 6 are continuity in x and (a) of the preceding problem.] 


For testing the hypothesis H’: 0, < 0 < 6, (0, < 0,) against the alternatives 
0 < 0, or 0 > @,, or the hypothesis 6 = 6, against the alternatives 0 + b, in 
an exponential family or more generally in a family of distributions satisfying 
the assumptions of Problem 30, a UMP test does not exist. 

[This follows from a consideration of the UMP tests for the one-sided 
hypotheses H; : 0 > 0, and H, : 0 < 0,.] 


Section 8 


Let the variables X, (i = 1,..., s) be independently distributed with Poisson 
distribution P(A,). For testing the hypothesis H:LA, < a (for example, that 
the combined radioactivity of a number of pitces of radioactive material does 
not exceed a), there exists a UMP test, which rejects when X X, > C. 

[If the joint distribution of the X’s is factored into the marginal distribution of 
XX, (Poisson with mean YA;) times the conditional distribution of the vari- 
ables Y, = X,/YX, given EX, (multinomial with probabilities p, = X,/XMN) 
the argument is analogous to that given in Example 8.] 


Confidence bounds for a median. Let X,,..., X, be a sample from a continu- 
ous cumulative distribution function F. Let £ be the unique median of F ifit 
exists, or more generally let = inf(£': F(é’) = 1). 


(i) If the ordered X's are Xa) < --- < Xim» a uniformly most accurate 
lower confidence bound for £ is £ = X4) with probability p, £ = Xu.«n 
with probability 1 — p, where k and p are determined by ` 


E()ze-2 È (p-e 


j=k+1 
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34. 


35. 


(ii) This bound has confidence coefficient 1 — a for any median of F. 


(iii) Determine most accurate lower confidence bounds for the 100p-per- 
centile £ of F defined by £ = inf(£': F(t’) =p}. 


[For fixed £y the problem of testing H: £ = £y against K: £ > £o is equivalent 
to testing H’: p = 1 against K': p < 1] 


A counterexample. Typically, as a varies the most powerful level-a tests for 
testing a hypothesis H against a simple alternative are nested in the sense that 
the associated rejection regions, say R,, satisfy Ra C Ra for any a < a’, This 
relation always holds when H is simple, but the following example shows that 
it need not be satisfied for composite H. Let X take on the values 1,2,3,4 
with probabilities under distributions Py, P}, Q: 


Then the most powerful test for testing the hypothesis that the distribution of 
X is P, or P, against the alternative that it is Q rejects at level a = i; when 
X — 1 or 3, and at level a = $ when X = 1 or 2. 


Let X and Y be the number of successes in two sets of n binomial trials with 
probabilities p, and p; of success. 


G) The most powerful test of the hypothesis H: p; < Pi against an alterna- 
tive (pi, p5) with pj < p; and pj + p? = lat level a < 1 rejects when 
Y — X » C and with probability y when Y — X-C. 

Gi) This test is not UMP against the alternatives p < P2: 


IG): Take the distribution A assigning probability 1 to the point p, = p; = i 
as an a priori distribution over H. The most powerful test against (pi, ph) is 
then the one proposed above. To see that A is least favorable, consider the 
probability of rejection B( pi, P2) for Pı = P» = P- By symmetry this is given 
by 


2B( p.p) = P(IY - XI» C) + YP(IY - XI 7 C)- 


Let X, be 1 or 0 as the ith trial in the first series is a success or failure, and let 
Y, be defined analogously with respect to the second series. Then Y pt - 
X^. (Y, — X;), and the fact that 2B(p, P) attains its maximum for p = 2 can 
be proved by induction over n. i ? 
(ii): Since B( p, p) < a for p * 3, the power f (pi, P2) is. < a for alternatives 
Pi < p, sufficiently close to the line p; = P2: That the test is not UMP now 
follows from a comparison with $(x, y) = a.) 
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Sufficient statistics with nuisance parameters. 


(i) A statistic T is said to be partially sufficient for @ in the presence of a 
nuisance parameter 7 if the parameter space is the direct product of the 
set of possible ĝ- and 7-values, and if the following two conditions hold: 
(a) the conditional distribution given T = ¢ depends only on n; (b) the 
marginal distribution of T depends only on 6. If these conditions are 
satisfied, there exists a UMP test for testing the composite hypothesis 
H:0 —6, against the composite class of alternatives 0 = 0,, which 
depends only on T. 


Gi) Part (i) provides an alternative proof that the test of Example 8 is UMP. 


[Let o(t) be the most powerful level a test for testing 6, against 8, that 
depends only on t, let $(x) be any level-a test, and let (1) = E, [6(X)|t]. 
Since Ej Y(T) = E, , $(X), it follows that j is a level-a test of H and its 
power, and therefore the power of $, does not exceed the power of yo.] 

Note. For further discussion of this and related concepts of partial sufficiency 
see Dawid (1975), Sprott (1975), Basu (1978), and Barndorff-Nielsen (1978). 


Section 9 
Let X,,..., X, and ¥,,...,¥, be independent samples from N(£,1) and 
N(n,1), and consider the hypothesis H : ņ < £ against K: n > £. There exists 
a UMP test, and it rejects the hypothesis when Y — X is too large. 
[If $ <m is a particular alternative, the distribution assigning probability 1 to 
the point n = = (mé, + nn,)/(m + n) is least favorable.] 


Let X,,..., X; Yis...» Y, be independently, normally distributed with means 
£ and n, and variances o? and 7? respectively, and consider the hypothesis 
H:r < o against K: o < r. 


G) If £ and 7 are known, there exists a UMP test given by the rejection 
region L(Y, — n)?/X(X, - £)? > C. 
(ii) No UMP test exists when £ and 7 are unknown. 


Additional Problems 


Let Py, P,, P, be the probability distributions assigning to the i E 
the following probabilities: assigning to the integers 1,..., 


6 


02 01 0 92 
P, 06 05 08 02 01 -78 
P, 09 05 12 0 02 2 


Determine whether there exists a level- test of H: P = P, which is UMP 
against the alternatives P, and P, when (i) a = -01; (ii) a = .05; (iii) a = .07. 


3.10] PROBLEMS 123 


40. 


41. 


42. 


43. 


Let the distribution of X be given by 


x 0 1 2 3 
P,(X = x) D 28 9-28 TET) 


where 0 < @ < .1. For testing H: 0 — .05 against @ > .05 at level a = .05, 
determine which of the following tests (if any) is UMP: 


© $0) =1, $1) = $@) = $0) = 0; 

G) $) = 5, $0) = $(2 = 9) = 0; 

(ii) $3) =1, ¢@ 7 €) = $2) = 0. 
Let X,,..., X, be independently distributed, each uniformly over the integers 
1,2,..., 0. Determine whether there exists a UMP test for testing H : 0 = 6, at 
level 1/0/ against the alternatives (i) 0 > 65; (ii) 0 < 0; (iii) 0 # bo- 


Let X, be independently distributed as N(iA,1), i = 1,..., n. Show that there 
exists a UMP test of H:A <0 against K: A> 0, and determine it as 
explicitly as possible. 

Note. The following problems (and some of the Additional Problems in later 
chapters) refer to the gamma, Pareto, Weibull, and inverse Gaussian distribu- 
tions. For more information about these distributions, see Chapter 17, 19, 20, 
and 25 respectively of Johnson and Kotz (1970). 


Let X,,..., X, be a sample from the gamma distribution T(g, b) with density 


1 


TR s Uses Uta 


Show that there exist a UMP test for testing 


G) H:b < b, against b > by when g is known; 
Gi) H:g < g, against g > go when b is known. 


In each case give the form of the rejection region. 


A random variable X has the Pareto distribution P(c,7) if its density is 
ct /x1O<71<x,0<e 


() Show that this defines a probability density. 
(i) If X has distribution P(c, v), then Y = log X has exponen 
tion E(£, b) with £ = log 7, b= 1/c. 
Gii) If X,,..., X, is a sample from P(c, 7), 
UMP tests of (a) H:t=% against T * ù% 
H: c= cp, 7 = % against ¢ > Co, T< T- 


tial distribu- 


use (ii) and Problem 3 to obtain 
when b is known; (b) 
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A random variable X has the Weibull distribution W(b, c) if its density is 


c 
z(5) eU, x0, b,c20. 


() Show that this defines a probability density. 

(ii) If X,..., X, is a sample from W(b, c), with the shape parameter c 
known, show that there exists a UMP test of H: b < b; against b > bo 
and give its form. 


Consider a single observation X from W(1, c). 


(i) The family of distributions does not have monotone likelihood ratio in x. 


(ii) The most powerful test of H: c = 1 against c = 2 rejects when X < kı 
and when X > k;. Show how to determine k, and kz. 


(iii) Generalize (ii) to arbitrary alternatives c; > 1, and show that a UMP test 
of H : c = 1 against c > 1 does not exist. 

(iv) For any c > 1, the power function of the MP test of H : c = 1 against 
c = c, is an increasing function of c. 


Let X,,..., X, be a sample from the inverse Gaussian distribution I(p, 7) 
with density 


T T 2 
3 exp -zge-»). x»0, r,u» 0. 


Show that there exists a UMP test for testing 


(i) H:p < po against p > po when r is known; 

(i) H:T < against 7 > 7, when p is known. 

In each case give the form of the rejection region. 

(iii) The distribution of V — (X, — p)?/X;p? is x?, and hence that of 
TE[CX, — n)/ Xi] is x2. 

[Let Y = min(X,, i2/X), Z- «(Y — p)?/w?Y. Then Z- V and Z is x? 

[Shuster (1968)]] 

Note. The UMP test for (ii) is discussed in Chhikara and Folks (1976). 

Let X be distributed according to Pj, 0 € Q, and let T be sufficient for 0. If 

9 (X) is any test of a hypothesis concerning 6, then (T) given by ¥(t) = 


E[q( X)|t] is a test depending on T only, an its power function is identical 
with that of p(X). 
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49. In the notation of Section 2, consider the problem of testing Hy: P = P; 
against H, : P = P,, and suppose that known probabilities mọ = 7 and 7 = 
1 — v can be assigned to Hy and H, prior to the experiment. 


(i) The overall probability of an error resulting from the use of a test is 


vEg(X) + (1- 7) E[1 - 9(X)]. 


(i) The Bayes test minimizing this probability is given by (8) with k = 79/71. 
(ii) The conditional probability of H; given X = x, the posterior probability 
of H, is 
7, p(x) 
MPo(x) + mp)" 


and the Bayes test therefore decides in favor of the hypothesis with the 
larger posterior probability. 

50. (i) For testing Hy: 0 = 0 against H; : 8 = 0, when X is N(0, 1), given any 
0 < a < 1 and any 0 < 7 < 1 (in the notation of the preceding problem), 
there exists 0, and x such that (a) Hp is rejected when X — x but (b) 
P(H,|x) is arbitrarily close to 1. 

(i) The paradox of part (i) is due to the fact that a is held constant while the 
power against 0, is permitted to get arbitrarily close to 1. The paradox 
disappears if a is determined so that the probabilities of type I and type 
II error are equal [but see Berger and Sellke (1984)]. 


[For a discussion of such paradoxes, see Lindley (1957), Bartlett (1957) and 
Schafer (1982).] 

51. Let X,,..., X, be iid. with density pp or pı, so that the MP level-a test of 
H: po rejects when IT ,r(X,) = C,, where r(X,) = pı(X;)/Po( X), or equiv- 
alently when 


(34) Fe ( Lo) - Follogr(X)]} = he 


G) It follows from the central limit theorem (Chapter 5, Theorem 3) that 
under H the left side of (34) tends in law to N(0, c?) with o? = 
Var, [log r( X;)] provided g? < oo. 

Gi) From (i) it follows that k, > ou, where ®(u,) = l-a. 

(ii) The power of the test (34) agaisnt P; tends to 1 as n > oo. 


[Gii): Problem 14(iv).] 


52. Let X,,..., X, be independent N(9, y), 0<y<1 known, and Y,,..., Y, 
independent N(8,1). Then X is more informative than Y according to the 


definition at the end of Section 4. 
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[If V, is N(0,1 — y), then X, + V, has the same distribution as Y,.] 

Note. If ø is unknown, it is not true that a sample from N(6, yo), 
0 <y<1, is more informative than one from N(6, o?); see Hansen ad 
Torgersen (1974). 


53. Let f,g be two probability densities with respect to p. For testing the 
hypothesis H:6 «6, or 62 0, (0 « 6, « 0, < 1) against the alternatives 
0) « 0 « 0, in the family P= (0f(x) + (1 — 0)g(x), 0 < 0 < 1), the test 
(x) = a is UMP at level a. 


11. REFERENCES 


Hypothesis testing developed gradually, with early instances frequently 
being rather vague statements of the significance or nonsignificance of a set 
of observations. Isolated applications are found in the 18th century 
[Arbuthnot (1710), Daniel Bernoulli (1734), and Laplace (1773), for exam- 
ple] and centuries earlier in the Royal Mint's Trial of the Pyx [discussed by 
Stigler (1977)]. They became more frequent in the 19th century in the 
writings of such authors as Gavarret (1840), Lexis (1875, 1877), and 
Edgeworth (1885). Systematic use of hypothesis testing began with the work 
of Karl Pearson, particularly his x? paper of 1900. 

The first authors to recognize that the rational choice of a test must 
involve consideration not only of the hypothesis but also of the alternatives 
against which it is being tested were Neyman and E. S. Pearson (1928). They 
introduced the distinction between errors of the first and second kind, and 
thereby motivated their proposal of the likelihood-ratio criterion as a 
general method of test construction. These considerations were carried to 
their logical conclusion by Neyman and Pearson in their paper of 1933, in 
which they developed the theory of UMP tests. Accounts of their collabora- 
tion can be found in Pearson's recollections (1966), and in the biography of 
Neyman by Reid (1982). 
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out for the first time by E. B. Wilson (1927). About the same time two 
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CHAPTER 4 


Unbiasedness: Theory and 
First Applications 


1. UNBIASEDNESS FOR HYPOTHESIS TESTING 


A simple condition that one may wish to impose on tests of the hypothesis 
H:@€Q,, against the composite class of alternatives K: 6 € Q x is that 
for no alternative in K should the probability of rejection be less than the 
size of the test. Unless this condition is Satisfied, there will exist alternatives 
under which acceptance of the hypothesis is more likely than in some cases 
in which the hypothesis is true. A test @ for which the above condition 
holds, that is, for which the power function B,(8) = E,9( X) satisfies 


B,(0)x«a if 6cQ,, 
(1) 
B,((0)2«a if 6€Q,, 


is said to be unbiased. For an appropriate loss function this was seen in 
Chapter 1 to be a particular case of the general definition of unbiasedness 
given there. Whenever a UMP test exists, it is unbiased, since its power 
cannot fall below that of the test $(x) = a. 

For a large class of problems for which a UMP test does not exist, there 
does exist a UMP unbiased test. This is the case in particular for certain 
hypotheses of the form 0 < O or 0 =, where the distribution of the 
tandom observables depends on other parameters besides 0. 

When B, (8) is a continuous function of 6, unbiasedness implies 


Q) B(8)=a forall 6 ino, 


where o is the common boundary of Q, and Q x; that is, the set of points 0 
that are points or limit points of both Q, and Qx. Tests satisfying this 
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condition are said to be similar on the boundary (of H and K). Since it is 
more convenient to work with (2) than with (1), the following lemma plays 
an important role in the determination of UMP unbiased tests. 


Lemma 1. Jf the distributions P, are such that the power function of every 
test is continuous, and if y is UMP among all tests Satisfying (2) and is a 
level-a test of H, then is UMP unbiased. 


Proof. The class of tests satisfying (2) contains the class of unbiased 
tests, and hence $, is uniformly at least as powerful as any unbiased test. 
On the other hand, ¢ is unbiased, since it is uniformly at least as powerful 
as ó(x) = a. 


2. ONE-PARAMETER EXPONENTIAL FAMILIES 


Let @ be a real parameter, and X = (X,,..., X,) a random vector with 
probability density (with respect to some measure p) 


Po(x) = C(0)e*" ?h(x). 


It was seen in Chapter 3 that a UMP test exists when the hypothesis H and 
the class K of alternatives are given by (i) H : 0 < b, K: 6 > 6, (Corollary 
2) and (ii) H : 8 < 6, or 8 > 6, (6, < 0,), K : 6, < 0 < 0, (Theorem 6), but 
not for (iii) H: 0, < 0 < 6, K:0 < 0, or 0 > @,. We shall now show that 
in case (iii) there does exist a UMP unbiased test given by 


1 when T(x)« Gor >Q, 
(3) $(x)-iw when T(x)- G, i=1,2, 
0 when C, < T(x) « C, 


where the C's and y’s are determined by 
(4) E,g(X) = Eyg(X) = a. 


The power function Ej$( X) is continuous by Theorem 9 of Chapter 2, 
so that Lemma 1 is applicable. The set w consists of the two points 0, and 
9,, and we therefore consider first the problem of maximizing Eq o(X) for 

` Some 6’ outside the interval [0,, 6], subject to (4). if this problem is 
restated in terms of 1 — (x), it follows from part (ii) of Theorem 6, 
Chapter 3, that its solution is given by (3) and (4). This test is therefore 
UMP among those satisfying (4), and hence UMP unbiased by Lemma 1. It 
further follows from part (iii) of the theorem that the power function of the 
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test has a minimum at a point between 6, and @,, and is strictly increasing 
as @ tends away from this minimum in either direction. 

A closely related problem is that of testing (iv) H: 0 = 6, against the 
alternatives 0 + 6,. For this there also exists a UMP unbiased test given by 
(3), but the constants are now determined by 


(5) E,,[¢(X)] =a 
and 
(6) E, [T(X)e(X)] = E, [T( X)]a. 


To see this, let 6’ be any particular alternative, and restrict attention to 
the sufficient statistic T, the distribution of which by Chapter 2, Lemma 8, is 
of the form 


aP,(t) = C(8) e” av(:). 


Unbiasedness of a test (1) implies (5) with $(x) = v[T(x)]; also that the 
power function &(0) = E,[¥(T)] must have a minimum at @ = 6j. By 
Theorem 9 of Chapter 2 the function B(8) is differentiable, and the 
derivative can be computed by differentiating EjJ/(T) under the expecta- 
tion sign, so that for all tests y(t) 


3 C'(8 
PO - &(rv(r)] + EO p, uer). 
For /(t) = a, this equation becomes 
C'(8 
0- E,(T) + cm 
Substituting this in the expression for B’(8) gives 


&'() = E,[Ty(T)] - E(T)Ely¥(T)], 


and hence unbiasedness impli i iti 
plies (6) in addition to (5) 
Let M be the set of poi 1 an 
totality of critical actions, en itte As Nimes ver the 


(u, uE (T) with 0 <u < 1. It also contains points (a, u ) with u, > 
» 5 2 


there exist tests with Ej [V(T)] = a 
apter 3). Since similarly M contains 
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points (a, u,) with u, < aE, (T), the point (a, aE, (T)) is an inner point of 
M. Therefore, by Theorem 5(iv) of Chapter 3 there exist constants ky, kz 
and a test (t) satisfying (5) and (6) with (x) = y[T(x)) such that 
w(t) = 1 when 


C(&)(k, + kzt) e*t < C(6") ef” 
and therefore when 
4, * a3t < e", 


This region is either one-sided or the outside of an interval. By Theorem 2 
of Chapter 3 a one-sided test has a strictly monotone power function and 
therefore cannot satisfy (6). Thus y(t) is 1 when t < C or > C, and 
the most powerful test subject to (5) and (6) is given by (3) This 
test is unbiased, as is seen by comparing it with (x) = a. It is then also 
UMP unbiased, since the class of tests satisfying (5) and (6) includes the 
class of unbiased tests. 

A simplification of this test is possible if for 6 = 6, the distribution of T 
is symmetric about some point a, that is, if PQT«a-u)j- PT» a- 
u for all real u. Any test which is symmetric about a and satisfies (5) must 
also satisfy (6), since Eg [TY(T)] = Eo I(T — a)9(T)] + aE 4 (T) = aa 
= E,(T)a. The C's and y’s are therefore determined by 


a 
Py {T < Ci} + nA (T7 6) = 7’ 


C, 22a - C, DOSE" 


The above tests of the hypotheses 0, < 0 < 6, and @ = 6, are strictly 
unbiased in the sense that the power is > a for all alternatives 0. For the 
first of these tests, given by (3) and (4), strict unbiasedness is an immediate 
Consequence of Theorem 6(iii) of Chapter 3. This states in fact that the 
Power of the test has a minimum at a point 6; between 0, and 0, and 
increases strictly as 6 tends away from 4 in either direction. The second of 
the tests, determined by (3), (5), and (6), has a continuous power function 
with a minimum of a at @ = 6. Thus there exist 0, < 6, < 6; such that 
B(8,) = B(@,) = c where a < c < 1. The test therefore coincides with the 
UMP unbiased level-c test of the hypothesis 0; < 0 < 6,, and the power 
increases strictly as moves away from 6, in either direction. This proves 
the desired result. 
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Example 1. Binomial, Let X be the number of successes in n binomial trials 
with probability p of success. A theory to be tested assigns to p the value po, so that 
one wishes to test the hypothesis H: p = po. When rejecting H one will usually 
wish to state also whether p appears to be less or greater than po. If, however, the 
conclusion that p * pọ in any case requires further investigation, the preliminary 
decision is essentially between the two possibilities that the data do or do not 
contradict the hypothesis p — py. The formulation of the problem as one of 
hypothesis testing may then be appropriate. 

The UMP unbiased test of H is given by (3) with T(X) = X. The condition (5) 
becomes 


€;-1 2 
-x n P 
Y (Jm«--xa- w( c) 964 Chea 
x=C,+1 i=l 


and the left-hand side of this can be obtained from tables of the individual 
probabilities and cumulative distribution function of X. The condition (6), with the 
help of the identity 


x(%) péqe-* = v" at Tia ay sae 


reduces to 
€;-1 : 
23» pns ) X-15(n-1)-(x-1) 
xeu xl i54 


2 
*La = w c mAT =1-a, 


the Gene side of Pe can be computed from the binomial tables. 
_ AS n increases, the distribution of (X — npg) yn tend: 

distribution N(0,1). For sample sizes which Ld oe coal ie op 

which are not too close to 0 or 1, the dis 1 s 


h T tribution of X is therefore aj i J 
symmetric, A P 3s e approximately 
y’s are enin Oy the much simpler “equal tails" test, for which the C's and 


Ci 
E (toa v(2 opa 


x-0 


n n 
-x(&)rta-o + L (saai 
mecs 


is approximately unb; 


unbiased test. Of course, ee nis omen reasonable approximation to the 


termined directly from the normal distribution. Rc Deni be de 
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Example 2. Normal variance. Let X — (3,,..., X,) be a sample from a nor- 
mal distribution with mean 0 and variance o » So that the density of the X’s is 


(ss on) 


Then T(x) = Ex? is sufficient for c, and has probability density (1/6?) f, ( y/6?), 
where 


1 
= — y(1/2)- 1,7 (972) 
f(y) 27T(n/2)" e , y» 0, 


is the density of a x?-distribution with n degrees of freedom. For varying o, these 
distributions form an exponential family, which arises also in problems of life 
testing (see Problem 14 of Chapter 2), and concerning normally distributed variables 
with unknown mean and variance (Section 3 of Chapter 5). The acceptance region 
of the UMP unbiased test of the hypothesis H : o = op is 
2 
Xi 
Cusb-sC 
I» 
with 
c. 
[ fo) =1-4 
€ 


and 


( - a) IER — 


f EO dy - 


For the determination of the constants from tables of the x?-distribution, it is 
convenient to use the identity 


Wal Y) = nf), 


to rewrite the second condition as 
MOr 7 17e. 
1 


Alternatively, one can integrate /£:yf, (y) dy by parts to reduce the second condi- 
on to 


Cs eg o7 = co e €, 
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[For tables giving C, and C, see Pachares (1961).] Actually, unless n is very small 
or a, very close to 0 or oo, the equal-tails test given by 


Ci F ae 
[5 a= hy) d= 5 


is a good approximation to the unbiased test. This follows from the fact that Tq 
suitably normalized, tends to be normally and hence symmetrically distributed for 
large n. 


UMP unbiased tests of the hypotheses (ii) H: 0, < 0 € 0, and (iv) 
H: 0 = 6, against two-sided alternatives exist not only when the family 
P(x) is exponential but also more generally when it is strictly totally 
positive (STP,,). A proof of (iv) in this case is given in Brown, Johnstone, 
and MacGibbon (1981); the proof of (iii) follows from Chapter 3, Problem 
30. 


3. SIMILARITY AND COMPLETENESS 


In many important testing problems, the hypothesis concerns a single 
real-valued parameter, but the distribution of the observable random vari- 
ables depends in addition on certain nuisance parameters. For a large class 
of such problems a UMP unbiased test exists and can be found through the 


Ej$(X) =a 


» for all distributions of X belonging to a gi ily 2X 

given family 2X = ( P, 9 € w}. 
Such tests are called similar with respect to PX or w, E. if $ i 
ae with critical region S, the latter is “simi 
space" Z in that both the probability P,( X € 
independent of 0 € w, E ERE e 


Let T be a sufficient statistic for 2X and let 27 i 
à suffici $ t P” denote the family { P7, 
9 € w) of distributions of T as 0 ranges over o. Then any test CH 


(7) E[e(X)]-«a ae. pr 
is similar with respect to P* since then 
E,[¢(X)] = EÉ(E[$(X)i]) =a forall 6€. 


*A statement is said to if i 
ree hold a.e. 9 if it holds except on a set W with P(N) = 0 for all 
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A test satisfying (7) is said to have Neyman structure with respect to T. It is 
characterized by the fact that the conditional probability of rejection is a on 
each of the surfaces T = t; Since the distribution on each such surface is 
independent of @ for 0 € €, the condition (7) essentially reduces the 


(8) Ep[f(X)]=0 forall Peg 
implies 
(9) f(x)=0 ae. 


In applications, # will be the family of distributions of a sufficient statistic. 
LI 


Example 3. Consider n independent trials with probability p of success, and let 
X, be 1 or 0 as the ith trial is a success or failure. Then Dex d wot X, dua 
sufficient statistic for D, and the family of its possible distributions is 2 = (bp, n), 
0 <p < 1). For this family (8) implies that 


Eno(1)s-o forall 0 « p< c, 


Where p = p/(1 — p), The left-hand side is a polynomial in p, all the coefficients of 
Which must be zero. Hence f(t) = 0 for t — 0,..., n and the binomial family of 
distributions of T is complete. 


Example 4. Let X, » X, be a sample from the uniform distribution U(0, 0), 
0 « 6x9 Then T- max(X;,..., X,) is a sufficient statistic for 6, and (8) 
becomes 


IO dPf Cr) = nd" f'f(o) n aie o forall 6. 


Let f(t) = f(t) — f (t) where f* and f^ denote the positive and negative parts of 
Í respectively, Then 


»*(A) -froena and » (A) -[roeta 
A 


are two measures over the Borel sets on (0, 00), which agree for all intervals and 
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hence for all A. This implies ft (t) = f (t) except possibly on a set of Lebesgue 
measure zero, and hence f(t) = 0 a.e. Z7. 


Example 5. Let X,,..., X,; %j,...,¥, be independently normally distributed 
as N(£, o^) and N(£, 7?) respectively. Then the joint density of the variables is 


1 £ 1 £ 
C(E, 0, ren -E + z2x s PERDA eR 5r». 
The statistic 
T-(Ex.XX.YY.YYy) 


is sufficient; it is, however, not complete, since E(XY/n — X.X,/m) is identically 
zero. If the Y's are instead distributed with a mean E(Y)- 3 which varies 
independently of £, the set of possible values of the parameters 0, = —1/207, 
9, = £/o?, 0, = —1/27?, 0, = 4/7? contains a four-dimensional rectangle, and it 
follows from Theorem 1 below that 27 is complete. 


Completeness of a large class of families of distributions including that of 
Example 3 is covered by the following theorem. 


Theorem 1. Let X be a random vector with probability distribution 
k 
dPi(x)  C(0)exp| X 6T(x) | du(x), 
j=l 


and let P” be the family of distributions of T = (T(X),..., T,(X)) as 0 


ranges over the set w. Then PT is complete provided « contains a k-dimen- 
sional rectangle. 


1 Proof. By making a translation of the parameter space one can assume 
without loss of generality that w contains the rectangle 


T= ((6,...,6):-a&6,«a, J9 lsk). 
Let f(t) = f*(t) — f-(t) be such that 


Eof(T)=0 forall 0 € c. 


Then for all @ € J, 


if v denotes the measure induced in T-space by the 
measure p, 


Jro dv(t) = f esr q) dv(t) 
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and hence in particular 


JEO avte) = fE) avo). 


Dividing f by a constant, one can take the common value of these two 
integrals to be 1, so that 


aP*(t) = ft (t) dv(t) and dP-(t) =f (t) dv(t) 


are probability measures, and 
fem dP*(t) = fer dP- (t) 


for all 0 in J. Changing the point of view, consider these integrals now as 
functions of the complex variables 9, - £ tin, J =1,...,k. For any 
fixed 6,,..., i-is b41 -- +» 9,, with real parts strictly between —a and +a, 
they are by Theorem 9 of Chapter 2 analytic functions of 9, in the strip 
Rj:-a<é,<a, —o <1; < oo of the complex plane. For ,,..., 0, 
fixed, real, and between —a and a, equality of the integrals holds on the 
line segment (&,m):-a«& <a, m= 0} and can therefore be ex- 
tended to the strip R}, in which the integrals are analytic. By induction the 
equality can be extended to the complex region ((8,, ..., 0,): ($n nj) ER, 
fOr, = 1]... k}. It follows in particular that for all real (05... 7) 


fet dP* (1) = fet dP" (t). 


These integrals are the characteristic functions of the distributions Pip and 
P- respectively, and by the uniqueness theorem for characteristic functions,* 
the two distributions P+ and P- coincide. From the definition of these 
distributions it then follows that f*(t) =f (t), a.e. v, and hence that 
F(t) = 0 ae. PT, as was to be proved. 

Example 6. Nonparametric completeness. Let X,,..., Xy be independently 
and identically distributed with cumulative distribution function F € F, where F 
is the family of all absolutely continuous distributions. Then the set of order 
Statistics T(X) = (Xq),.--, Xyy) was shown to be sufficient for F in Chapter 2, 
Section 6. We shall now prove it to be complete. Since, by Example 7 of Chapter 2, 
T'(x)- (ZX, X X2,..., EXN) is equivalent to T( X) in the sense that both induce 
the same subfield of the sample space, T'(X) is also sufficient and is complete if 


*See for example Section 26 of Billingsley (1979). 
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and only if T( X) is complete. To prove the completeness of T’(X) and thereby that 
of T(X), consider the family of densities 


f(x) = C(6,,..., Oy Jexp(—x?" + Ox + +Oyx"), 


where C is a normalizing constant. These densities are defined for all values of the 
6's since the integral of the exponential is finite, and their distributions belong to 7. 
The density of a sample of size N is 


C'exp( - Exi! + Dx + 0° t Y xN) 


and these densities constitute an exponential family A%. By Theorem 1, T'( X) is 
complete for A, and hence also for F, as was to be proved. 

The same method of proof establishes also the following more general result. Let 
X, j 7l... Np i= 1,..., c, be independently distributed with absolutely con- 
tinuous distributions F, and let X? < --- < X("? denote the N, observations 
Xa. .-, Xin, arranged in increasing order. Then the set of order statistics 


(9,..., MIM, 30,..., XP) 


is sufficient and complete for the family of distributions obtained by letting 

F,,..., F. range over all distributions of #. Here completeness is proved by 

ADI. the subfamily % of F in which the distributions F, have densities of 
e form 


f(x) = C (0,,..., By, )exp( —x?% TO ax es 36, x"). 


_ The result remains true if F is replaced by the family F, of continuous 
distributions, For a proof see Problem 12 or Bell, Blackwell, and Breiman (1960). 


For the present purpose the slightly weaker property of bounded com- 
pleteness is appropriate, a family P of probability distributions being 
boundedly complete if for all bounded functions f, (8) implies (9). If P is 
complete it is a fortiori boundedly complete. 


Theorem 2. Let x be a random variable with distribution P € P, and let 
T be a sufficient statistic for P. Then a necessary and sufficient condition for 


all similar tests to have Neyman structure with res, ect to T i. 1 
P” of distributions of T is boundedly complete. (COP S Ed 


Proof. Suppose first that PT is bounded! 
similar with séepectto d. Thon ly complete, and let $( X) be 


E[€(X)-a] -0 forall Peg 
and hence, if y(t) denotes the conditional expectation of ó( X) — a given t, 
EYV(T)=0 forall PTE QT. 
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Since y(t) can be taken to be bounded by Lemma 3 of Chapter 2, it follows 
from the bounded completeness of 27 that y(t) = 0 and hence E[$( X)|t] 
=a a.e. PT, as was to be proved. 

Conversely suppose that 27 is not boundedly complete. Then there 
exists a function f such that |f(t)| € M for some M, that Ef(T ) = 0 for all 
PT € 27, and f(T) + 0 with positive probability for some P7 € PT, Let 
$(t) = of(t) + a, where c = min(a,1 — a)/M. Then ¢ is a critical func- 
tion, since 0 < $(t) < 1, and it is a similar test, since EQ(T) = a for all 
P" € 27. But $ does not have Neyman structure, since (T) * a with 
positive probability for at least some distribution in 7. 


4. UMP UNBIASED TESTS FOR MULTIPARAMETER 
EXPONENTIAL FAMILIES 


An important class of hypotheses concerns a real-valued parameter in an 
exponential family, with the remaining parameters occurring as unspecified 
nuisance parameters. In many of these cases, UMP unbiased tests exist and 
can be constructed by means of the theory of the preceding section. 

Let X be distributed according to 


(10) 
dPj (x) = C(0, oye] ou) + Eon) du(x), (0,9)€8Q, 


and let = (2,,..., 0,) and T = (7,,..., Ty). We shall consider the prob- 
lems* of testing the following hypotheses H, against the alternatives K,, 
Jes 13055, 4: 


H,:0 & 6; K,:9>% 

H,: 0 < 0, orð > 0, K;,:0, «0 «0, 
H,: 0, <0 <0, K,:0 « 0, orð 7 0, 
H,: 0 — 6, K,:0* 6. 


We shall assume that the parameter space Q is convex, and that it has 
dimension k + l, that is, that it is not contained in a linear space of 
dimension < k + 1. This is the case in particular when Q is the natural 
Parameter space of the exponential family. We shall also assume that there 
are points in Q with ð both < and > 65, 6,, and 6, respectively. 


"Such problems are also treated in Johansen (1979), which in addition discusses large- 
Sample tests of hypotheses specifying more than one parameter. 
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Attention can be restricted to the sufficient statistics (U, T) which have 
the joint distribution 


(11) aP y(u, t) = C(8, deo + Eos] dv(u,t), (0,8)cQ. 
i=l 


When T = t is given, U is the only remaining variable and by Lemma 8 of 
Chapter 2 the conditional distribution of U given t constitutes an exponen- 
tial family 


dPj" (u) = C(0)e*" dy,(u). 


In this conditional situation there exists by Corollary 2 of Chapter 3 a UMP 
test for testing H, with critical function $, satisfying 


1 when u Q(t), 
(12) $(u,1) 2 y(t) when u= Cr), 
0 when u< Q(t), 


where the functions C) and Yo are determined by 
(13) E,,[¢(U,T)|t] =a forall t. 


For testing H, in the conditional family there exists by Theorem 6 of 
Chapter 3 a UMP test with critical function 


1 when Ci(t) < u « C(t), 
(14) o(u,t)=(7(t) when u= C(t), i=1,2, 
0 when u<C,(t) or > C(t), 


where the C’s and y’s are determined by 
(15) F,,[¢(U, T)r] = E, [o (U, T Ui] = a. 


Consider next the test $; satisfying 


1 when u« Ci(t)or » C à 
(16) $(wr) - (y(t) when u= bM - e 


0 when Cir) <u < C,(r), 
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with the C's and y's determined by 


(17) E, [6(U, T)|t] = Ej, [64(U, T)|t] = a. 


When T = t is given, this is (by Section 2 of the present chapter) UMP 
unbiased for testing H, and UMP among all tests satisfying (17). 

Finally, let , be a critical function satisfying (16) with the C's and y's 
determined by 


(18) Ey,[¢4(U, T)|t] =a 
and 
(19) Es, [Uġ4(U, T)|t] = aE, [U]]. 


Then given T = t, it follows again from the results of Section 2 that $, is 
UMP unbiased for testing H, and UMP among all tests satisfying (18) and 
(19). 

So far, the critical functions $; have been considered as conditional tests 
given T = f. Reinterpreting them now as tests depending on U and T for 
the hypotheses concerning the distribution of X (or the joint distribution of 
U and T) as originally stated, we have the following main theorem.* 


Theorem 3. Define the critical functions $ by (12) and (13); $, by (14) 
and (15); $, by (16) and (17); $, by (16), (18), and (19). These constitute 
UMP unbiased level- tests for testing the hypotheses H,,..., H, respectively 
when the joint distribution of U and T is given by (11). 


Proof. The statistic T is sufficient for 9 if @ has any fixed value, and 
hence T is sufficient for each 


w= {(6,9):(0,8)€2,0=8}, jf =0,1,2. 


By Lemma 8 of Chapter 2, the associated family of distributions of T is 
given by 


k 
aPy.a(t) = C(8, aJe L Z dvlt), (6,9)&«,  j=0,1,2. 
i=l 


Since by assumption Q is convex and of dimension k + 1 and contains 


*A somewhat different asymptotic optimality property of these tests is established by 
Michel (1979). 
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points on both sides of 0 = 6,, it follows that c is convex and of dimension 
k. Thus w; contains a k-dimensional rectangle; by Theorem 1 the family 


97 = { PF ,:(0,8) eu} 
is complete; and similarity of a test ó on w, implies 


Ey[o(U,7)|] = a. 


(1) Consider first H,. By Theorem 9 of Chapter 2 the power function of 
all tests is continuous for an exponential family. It is therefore enough to 
prove $; to be UMP among all tests that are similar on «y (Lemma 1), and 
hence among those satisfying (13). On the other hand, the overall power of a 
test $ against an alternative (0, #) is 


OO &le(. T) = f| fot anro] arr) 


One therefore maximizes the overall power by maximizing the power of the 
conditional test, given by the expression in brackets, separately for each t. 
Since $, has the property of maximizing the conditional power against any 
0 > 6, subject to (13), this establishes the desired result. 

(2) The proof for H, and H, is completely analogous. By Lemma 1, it 
is enough to prove 9; and $, to be UMP among all tests that are similar on 
both w, and w,, and hence among all tests satisfying (15). For each t, $2 
and $, maximize the conditional power for their respective problems 
subject to this condition and therefore also the unconditional power. 

(3) Unbiasedness of a test of H, implies similarity on w and 


à 
ga [£0.06 (U, T)-0 on. 


The differentiation on the left-hand side of this equation can be carried out 


under the expectation sign, and by the computation which earlier led to (6), 
the equation is seen to be equivalent to 


E, ,[U$(U, T) Sag] — 9. Von wy. 


Therefore, since PI is complete, unbiasedness implies (18) and (19). As in 
the preceding cases, the test, which in addition satisfies (16), is UMP among 
all tests satisfying these two conditions. That it is UMP unbiased now 
follows, as in the proof of Lemma 1, by comparison with the test $(u, t) = a. 


4.4] MULTIPARAMETER EXPONENTIAL FAMILIES 149 


(4) The functions $,,..., $, were obtained above for each fixed t as a 
function of u. To complete the proof it is necessary to show that they are 
jointly measurable in u and 1, so that the expectation (20) exists. We shall 
prove this here for the case of $;; the proof for the other cases is sketched in 
Problems 14 and 15. To establish the measurability of ¢,, one needs to show 
that the functions C)(t) and y(t) defined by (12) and (13) are -measur- 
able. Omitting the subscript 0, and denoting the conditional distribution 
function of U given T = t and for 0 = @ by 


F(u) = P {U < ult}, 
one can rewrite (13) as 
F(C) - Y[E(C) - F(C - 0)] = 1 - a. 
Here C = C(t) is such that F(C — 0) < 1 — a < F(C), and hence 
C(t) =F (1 - a) 


where F- }(y) = inf(u: F(u) > y}. It follows that C(t) and y(t) will both 
be measurable provided F(u) and F(u — 0) are jointly measurable in u 
and ¢ and F; }(1 — a) is measurable in t. 


For each fixed u the function F,(u) is a measurable function of £, and for 
each fixed ; it is a cumulative distribution function and therefore in 
Particular nondecreasing and continuous on the right. From the second 
property it follows that F(u) > c if and only if for each n there exists a 
rational number r such that u € r < u + 1/n and F/(r) > c. Therefore, if 
the rationals are denoted by ry Tna 


1 
((u, 2): F(u) zc) - QU[(G. 0:087 «7. F(r;) 2c. 


This shows that F(u) is jointly measurable in u and t. The proof for 
F(u — 0) is completely analogous. Since F; (y) < u if and only if F(u) > 
Y, F; (y) is t-measurable for any fixed y and this completes the proof. 
The test @, of the above theorem is also UMP unbiased if Q is replaced 
by the set Q^ = QN ((0, 9): 0 > 6,), and hence for testing H':0 =b 
against à > 6. The assumption that Q should contain points with 0 < 4% 
was in fact used only to prove that the boundary set wọ contains a 
k-dimensional rectangle, and this remains valid if € is replaced by g. 
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The remainder of this chapter as well as the next chapter will be 
concerned mainly with applications of the preceding theorem to various 
statistical problems. While this provides the most expeditious proof that the 
tests in all these cases are UMP unbiased, there is available also a variation 
of the approach, which is more elementary. The proof of Theorem 3 is quite 
elementary except for the following points: (i) the fact that the conditional 
distributions of U given T = t constitute an exponential family, (ii) that the 
family of distributions of T is complete, (iii) that the derivative of 
E,9(U, T) exists and can be computed by differentiating under the 
expectation sign, (iv) that the functions $;,..., $, are measurable. Instead 
of verifying (i) through (iv) in general, as was done in the above proof, it is 
possible in applications of the theorem to check these conditions directly for 
each specific problem, which in some cases is quite easy. 

Through a transformation of parameters, Theorem 3 can be extended to 
cover hypotheses concerning parameters of the form 


k 
*=a0+ La,  agj*0. 


i=l 


This transformation is formally given by the following lemma, the proof of 
which is immediate. 


Lemma 2. The exponential family of distributions (10) can also be written 
as 


dPj (x) = K(0*, 9)exp|0*U*(x) + Y:9,7,*(x)] du(x) 


where 


Ue s di 
Roca uii is eU 
0 a 


Application of Theorem 3 to the form of the distributions given i 
n s given in the 
lemma leads to UMP unbiased tests of the hypothesis Hy¥: 6* z ) and the 
analogously defined hypotheses Hf, Hy, Hg. e fe 
When testing one of the hypotheses H, one is fr i i 
: : j equently interested in the 
power BO, 5) of $; against some alternative 0'. As is indicated by the 
notation and is seen from (20), this power will usually depend on 


the unknown nuisance parameters à. On th 
m : e other h: 
conditional test given T = 1, and, the power of the 


B(O"t) = Ep [e(U, T)], 


is independent of 9 and therefore has a known value. 
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The quantity B(0'|t) can be interpreted in two ways: (i) It is the 
probability of rejecting H when T = t. Once T has been observed to have 
the value 1, it may be felt, at least in certain problems, that this is a more 
appropriate expression of the power in the given situation than (6, 3), 
which is obtained by averaging 8(8’|t) with respect to other values of ¢ not 
relevant to the situation at hand. This argument leads to difficulties, since in 
many cases the conditioning could be carried even further and it is not clear 
where the process should stop. (ii) A more clear-cut interpretation is 
obtained by considering B(0/|t) as an estimate of B(0’, 9). Since 


Ev S [ B(9'IT)] = B(6", 9), 


this estimate is unbiased in the sense of Chapter 1, equation (11). It follows 
further from the theory of unbiased estimation and the completeness of the 
exponential family that among all unbiased estimates of 8(0', 9) the 
present one has the smallest variance. (See TPE, Chapter 2.) 

Regardless of the interpretation, B(@’|t) has the disadvantage compared 
with an unconditional power that it becomes available only after the 
Observations have been taken. It therefore cannot be used to plan the 
experiment and in particular to determine the sample size, if this must be 
done prior to the experiment. On the other hand, a simple sequential 
procedure guaranteeing a specified power f against the alternatives 0 = 0^ 
is obtained by continuing taking observations until the conditional power 
B(O'|t) is > B. 

The general question of whether to interpret measures of performance 
such as the power of a test or coverage probability of a family of confidence 
Statements conditionally, and if so, conditionally on what aspects of the 
data, will be considered in Chapter 10. 


5. COMPARING TWO POISSON OR BINOMIAL 
POPULATIONS 


A problem arising in many different contexts is the comparison of two 
treatments or of one treatment with a control situation in which no 
treatment is applied. If the observations consist of the number of successes 
ina sequence of trials for each treatment, for example the number of cures 
of a certain disease, the problem becomes that of testing the equality of two 
binomial probabilities. If the basic distributions are Poisson, for example in 
à comparison of the radioactivity of two substances, one will be testing the 
equality of two Poisson distributions. mah 
When testing whether a treatment has a beneficial effect by comparing it 
with the control situation of no treatment, the problem is of the one-sided 
type. If £, and £, denote the parameter values when the treatment is or is 
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not applied, the class of alternatives is K : $, > £,. The hypothesis is $, = £, 
if it is known a priori that there is either no effect or a beneficial one; it is 
£,& &, if the possibility is admitted that the treatment may actually be 
harmful. Since the test is the same for the two hypotheses, the second 
somewhat safer hypothesis would seem preferable in most cases. 

A one-sided formulation is sometimes appropriate also when a new 
treatment or process is being compared with a standard one, where the new 
treatment is of interest only if it presents an improvement. On the other 
hand, if the two treatments are on an equal footing, the hypothesis £; = £, 
of equality of two treatments is tested against the two-sided alternatives 
£; + Ej. The formulation of this problem as one of hypothesis testing is 
usually quite artificial, since in case of rejection of the hypothesis one will 
obviously wish to know which of the treatments is better.* Such two-sided 
tests do, however, have important applications to the problem of obtaining 
confidence limits for the extent by which one treatment is better than the 
other. They also arise when the parameter £ does not measure a treatment 
effect but refers to an auxiliary variable which one hopes can be ignored. 
For example, £, and $, may refer to the effect of two different hospitals in a 
medical investigation in which one would like to combine the patients into a 
single study group. (In this connection, see also Chapter 7, Section 3.) 

To apply Theorem 3 to this comparison problem it is necessary to express 
the distributions in an exponential form with 0 = f(£,, £;), for example 
0 = £, — & or §,/é,, such that the hypotheses of interest become equivalent 
to those of Theorem 3. In the present section the problem will be considered 
for Poisson and binomial distributions; the case of normal distributions will 
be taken up in Chapter 5. 

We consider first the Poisson problem in which X and Y are indepen- 


dently distributed according to P(A) and P(p), so that their joint distribu- 
tion can be written as 


e Atn) 


n. A u 
P{X=x,Y=y} eat Om ylog v + (x + y)log A]. 


By Theorem 3 there exist UMP unbiased tests of the four hypotheses 
Hy dy H, concerning the parameter 8 = log(u/A) or equivalently concern- 
ing S ratio p i gx This includes in particular the hypotheses p < A (or 
H = ^) against the alternatives p > À, and p = À against npari 

\) again v : Ii + À. Comparin 
the distribution of ( X, Y) with (10), one has U = Y andT=X+ Y, dia by 
Theorem 3 the tests are performed conditionally on the integer points of the 


*For a discussion of the comparison of tw isi 
itia usd ant LACRA ram o treatments as a three-decision problem, see 
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line segment X + Y = ¢ in the positive quadrant of the (x, y) plane. The 
conditional distribution of Y given X + Y = t is (Problem 13 of Chapter 2) 


t HS RA EC. 
P{Y=y|X+ r-0 - (55) [5] y y70,1,...,t, 


the binomial distribution corresponding to : trials and probability p = 
L/(X-- u) of success. The original hypotheses therefore reduce to the 
corresponding ones about the parameter p of a binomial distribution. The 
hypothesis H : p < aA, for example, becomes H: p < a/(a + 1), which is 
rejected when Y is too large. The cutoff point depends of course, in addition 
to a, also on t. It can be determined from tables of the binomial, and for 
large ¢ approximately from tables of the normal distribution. 

In many applications the ratio p = »/A is a reasonable measure of the 
extent to which the two Poisson populations differ, since the parameters À 
and » measure the rates (in time or space) at which two Poisson processes 
produce the events in question. One might therefore hope that the power of 
the above tests depends only on this ratio, but this is not the case. On the 
contrary, for each fixed value of p corresponding to an alternative to the 
hypothesis being tested, the power (À, u) = B(A, pA) is an increasing 
function of A, which tends to 1 as À — oo and to a as A > 0. To see this 
consider the power f(p|t) of the conditional test given t. This is an 
increasing function of t, since it is the power of the optimum test based on 1 
binomial trials. The conditioning variable T has a Poisson distribution with 
parameter A(1 + p), and its distribution for varying À forms an exponential 
family. It follows (Lemma 2 of Chapter 3) that the overall power E[(p|T)] 
is an increasing function of A. As A > 0 or co, T tends in probability to 0 
Or co, and the power against a fixed alternative p tends to a or 1. 

The above test is also applicable to samples X;,..., Xm and Y;,..., Y, 
from two Poisson distributions. The statistics X = L7L,X; and Y = Y Y, 
are then sufficient for À and p, and have Poisson distributions with 
parameters mÀ and ny respectively. In planning an experiment one might 
wish to determine m = n so large that the test of, say, H : p < po has power 
against a specified alternative p; greater than or equal to some preassigned 
B. However, it follows from the discussion of the power function for n — 1, 
which applies equally to any other n, that this cannot be achieved for any 
fixed n, no matter how large. This is seen more directly by noting that as 
A > 0, for both p = Po and p = p; the probability of the event Y= Y - 0 
tends to 1. Therefore, the power of any level-a test against p = p, and for 
varying À cannot be bounded away from a. This difficulty can be overcome 
only by permitting observations to be taken sequentially. One can for 
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example determine t, so large that the test of the hypothesis p < po/(1 + po) 
on the basis of tg binomial trials has power > B against the alternative 
Pı = py/ + (3). By observing (X;, Y1), (X2; Y;).... and continuing until 
X(X, + Y,) = to, one obtains a test with power > f against all alternatives 
with p > p,.* 

The corresponding comparison of two binomial probabilities is quite 
similar. Let X and Y be independent binomial variables with joint distribu- 
tion 


PC x Y7y) = (x) (S) pias” 
" (aret (oe - d 


+(x + yg] 
qı 


The four hypotheses H,,..., H4 can then be tested concerning the parame- 


ter 
0- i| nn ) 
hj f 


or equivalently concerning the odds ratio (also called cross-product ratio) 


P2 | Pr 
(ig ee eta 
aj a 


This includes in particular the problems of testing Hy: p, < p, against 
A ae Hg: py =p, against P2 * pı. As in the Poisson case, U = Y 
an = X + Y, and the test is carried out in terms of the conditional 


spine of Y on the line segment X + Y = t. This distribution is given 


(21 P{Y=y|X+Y= = w n 

EEPE E E A ymo 
*A discussion of this and alternati ievi 

MEL d. o! is and ali tive procedures for achieving the same aim is given by 
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where 


Cle) == 


1 
(G7) 


y=0 


In the particular case of the hypotheses Hý and Hj, the boundary value 6; 
of (13), (18), and (19) is 0, and the corresponding value of p is pọ = 1. The 
conditional distribution then reduces to 
m n 
(, Ei ylly ) 


P{Y=y|X+ al bl 
t 


which is the hypergeometric distribution. ; 

Tables of critical values by Finney (1948) are reprinted in Biometrika 
Tables for Statisticians, Vol. 1, Table 38 and are extended in Finney, 
Latscha, Bennett, Hsu, and Horst (1963, 1966). Somewhat different ranges 
are covered in Armsen (1955), and related charts are provided by Bross and 
Kasten (1957). Extensive tables of the hypergeometric distributions have 
been computed by Lieberman and Owen (1961). Various approximations 
are discussed in Johnson and Kotz (1969, Section 6.5) and by Ling and 
Pratt (1984); see also Cressie (1978). 

The UMP unbiased test of p, = Pz which is based on the (conditional) 
hypergeometric distribution, requires randomization to obtain an exact 
conditional level a for each f of the sufficient statistic T. Since in practice 
randomization is usually unacceptable, the one-sided test is frequently 
performed by rejecting when Y > C(T), where C(t) is the smallest integer 
for which P(Y > C(T)|T=t} <a. This conservative test is called 
Fisher’s exact test [after the treatment given in Fisher (1934)], since the 
probabilities are calculated from the exact hypergeometric rather than an 
approximate normal distribution. The resulting conditional levels (and 
hence the unconditional level) are often considerably smaller than a, and 
this results in a substantial loss of power. An approximate test whose overall 
level tends to be closer to a is obtained by using the normal approximation 
to the hypergeometric distribution without continuity correction. [For a 
comparison of this test with some competitors, see e.g. Garside and Mack 
(1976).] A nonrandomized test that provides a conservative overall level, but 
that is less conservative than the "exact" test, is described by Boschloo 
(1970) and by McDonald, Davis, and Milliken (1977). Convenient entries 
into the extensive literature on these and related aspects of 2 X 2 tables can 
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be found in Conover (1974), Kempthorne (1979), and Cox and Plackett 
(1980); see also Haber (1980), Barnard (1982), Overall and Starbuck (1983), 
and Yates (1984). For extensions to r X c tables, see Mehta and Patel 
(1983) and the literature cited there. 


6. TESTING FOR INDEPENDENCE IN A 2 x 2 TABLE 


The problem of deciding whether two characteristics A and B are indepen- 
dent in a population was discussed in Section 4 of Chapter 3 (Example 4), 
under the assumption that the marginal probabilities p(A) and p(B) are 
known. The most informative sample of size s was found to be one selected 
entirely from that one of the four categories A, A, B, or B, say A, which is 
rarest in the population. The problem then reduces to testing the hypothesis 
H: p = p(B) in a binomial distribution b( p, s). 

In the more usual situation that p( A) and p(B) are not known, a sample 
from one of the categories such as A does not provide a basis for dis- 
tinguishing between the hypothesis and the alternatives. This follows from 
the fact that the number in the sample possessing characteristic B then 
constitutes a binomial variable with probability p(B|A), which is com- 
pletely unknown both when the hypothesis is true and when it is false. The 
hypothesis can, however, be tested if samples are taken both from categories 
A and A or both from B and B. In the latter case, for example, if the 
sample sizes are m and n, the numbers of cases possessing characteristic A 
in the two samples constitute independent variables with binomial distri- 
butions b( p;, m) and b( p;, n) respectively, where Pı P(A|B) and p, = 
P(A|B). The hypothesis of independence of the two characteristics, p( 4|B) 
7 p(A), is then equivalent to the hypothesis p, — Pa, and the problem 
reduces to that treated in the preceding section. 

Instead of selecting samples from two of the categories, it is frequently 
more convenient to take the sample at random from the population as a 
whole. The results of such a'sample can be summarized in the following 


2 2 contingency table, the entries of which gi : 
various categories: give the numbers in the 
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The joint distribution of the variables X, X’, Y, and Y’ is multinomial, 
and is given by 


P{X=x, X’=x',Y=y,Y’=y’} 


s! 3 , 

ode yA PAnPAnPARPAR 

s! Pap Pip Pak 

= ———_—_ pi exp| x log —— + x’ log—— + y log—- |. 
xix’lyly PAR p Pih pn 


Lemma 2 and Theorem 3 are therefore applicable to any parameter of the 
form 


Pi Pab 
0* = alog ŽAB, alog = + ajlog —. 
PAB PAB PAB 


Putting a, — a; — 1, ag = —1, A — e" = (pigPag)/(PanPaa), and de- 
noting the probabilities of A and B in the population by p, = Paz + Pap, 
Pe = Pag + Pip, one finds 


1-A 
Pas = PaPe + A PásPAb: 


1-4 
Pain P4PB x PApPabs 


1-A 
Pab — PaP ON PánPAB: 


IPES 
Pap = PaP + — g PaBPab- 


Independence of A and B is therefore equivalent to A = 1, and A < 1 and 
A > 1 correspond to positive and negative dependence respectively." 

The test of the hypothesis of independence, or any of the four hypotheses 
concerning A, is carried out in terms of the conditional distribution of X 
given X-- X^— m, X4 Y — t. Instead of computing this distribution 


tA is equivalent to Yule’s measure of association, which is Q = (1 — A)/(1 + A). For a 
discussion of this and related measures see Goodman and Kruskal (1954, 1959), Edwards 


(1963), and Haberman (1982). 
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directly, consider first the conditional distribution subject only to the 
condition X + X’ = m, and hence Y + Y’ = s — m = n. This is seen to be 


P{X=x,Y=y|X+ X’ =m} 


E code are 
ATATA Pa} \ Pa 
which is the distribution of two independent binomial variables, the number 
of successes in m and n trials with probability p, = p,5/py and p, = 
P4p/Pj. Actually, this is clear without computation, since we are now 
dealing with samples of fixed size m and n from the subpopulations B and 
B, and the probability of A in these subpopulations is p; and p,. If now the 
additional restriction X + Y = t is imposed, the conditional distribution of 
X subject to the two conditions X + X^ = m and X + Y= t is the same as 


that of X given X+ Y — t in the case of two independent binomials 
considered in the previous section. It is therefore given by 


P(X-x|X* X!'2m, X+Y=1t) = GG(?(, * Jems, 


Emin t, 


that is, by (21) expressed in terms of x instead of y. (Here the choice of X 


as testing variable is quite arbitrary; we could equally well again have 
chosen Y.) For the parameter p one finds 


pud Pi. PaPas _ 4. 
421| d  PasPig 


From these considerations it follows that the conditional test given X + X’ 
=m, X + Y = t, for testing any of the hypotheses concerning A is identi- 
cal with the conditional test given X -- Y — t of the same hypothesis 
concerning p =A inthe Preceding section, in which X  X' 2 m was given 
à priori. In particular, the conditional test for testing the hypothesis of 
independence A = 1, Fisher's exact test, is the same as that of testing the 
equality of two binomial P's and is therefore Biven in terms of the 
hypergeometric distribution, 


, At the beginning of the section it was pointed out that the hypothesis of 
independence can be tested on the basis of samples obtained in a number of 
different ways. Either samples of fixed size can be taken from A and A or 
from B and B, or the sample can be selected at random from the 
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population at large. Which of these designs is most efficient depends on the 
cost of sampling from the various categories and from the population at 
large, and also on the cost of performing the necessary classification of a 
selected individual with respect to the characteristics in question. Suppose, 
however, for a moment that these considerations are neglected and that the 
designs are compared solely in terms of the power that the resulting tests 
achieve against a common alternative. Then the following results* can be 
shown to hold asymptotically as the total sample size s tends to infinity: 


(i) If samples of size m and n (m + n = s) are taken from B and B 
or from A and A, the best choice of m and n is m =n = s/2. 
(i) It is better to select samples of equal size s/2 from B and B than 
from A and A provided |p, — 3| > | pa — 41. 
(ii) Selecting the sample at random from the population at large is 
worse than taking equal samples either from A and A or from B 
and B. 


These statements, which we shall not prove here, can be established by 
using the normal approximation for the distribution of the binomial vari- 
ables X and Y when m and n are fixed, and by noting that under random 
sampling from the population at large, M/s and N/s tend in probability to 
Pz and pg respectively. 


7. ALTERNATIVE MODELS FOR 2 x 2 TABLES 


Conditioning of the multinomial model for the 2 x 2 table on the row (or 
column) totals was seen in the last section to lead to the two-binomial model 
of Section 5. Similarly, the multinomial model itself can be obtained as a 
conditional model in some situations in which not only the marginal totals 
M, N, T, and T’ are random but the total sample size s is also a random 
variable. Suppose that the occurrence of events (e.g. patients presenting 
themselves for treatment) is observed over a given period of time, and that 
the events belonging to each of the categories AB, AB, AB, AB are governed 
by independent Poisson processes, so that by (2) of Chapter 1 the num- 
bers X, X', Y, Y' are independent Poisson variables with expectations 
Naps À Am Naps Azz, and hence s is a Poisson variable with expectation 

= apt Àag t Àag t AaB / 

It may then be ef interet to compare the ratio \4,/A ;5 with À AM AB 
and in particular to test the hypothesis H : \ 45/A jg < À45/À5. The joint 
distribution of X, X", Y, Y’ constitutes a four-parameter exponential family, 


“These results were conjectured by Berkson and proved by Neyman in a course on x?. 
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which can be written as 
P(X=x, X'=x',Y=y,Y' =y’) 


1 Napr ap 
SoS 33x | Ct (x’ + x)log A; 
x!x'tyty't coe NIBA Ze (x + x)log ig 


+(y + x)log dys + (y — seg 


Thus, UMP unbiased tests exist of the usual one- and two-sided hypotheses 
concerning the parameter @ = À ,,À 4b/^ g^ 45. These are carried out in 
terms of the conditional distribution of X given 


X --X-m, Y+X=1, X+X+V+Y=s, 


where the last condition follows from the fact that given the first two it is 
equivalent to Y’ - X = s — t — m. By Problem 13 of Chapter 2, the condi- 
tional distribution of X, X',Y given X+ X'+Y+ Y'= s is the multi- 
nomial distribution of Section 6 with 


Nab Ag 
Bou PT MC a=. 


À 


and tf’ are therefore fixed, so that we are now 
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Then any one of the four cell counts X, X’, Y, Y’ determines the other three. 
Under H, the distribution of Y is the hypergeometric distribution derived as 
the conditional null distribution of Y given X + Y = 1 at the end of Section 
5. The hypothesis is rejected in favor of the alternative that treatment B 
enhances success if Y is sufficiently large. Although this is the natural test 
under the given circumstances, no optimum property can be claimed for it, 
since no clear alternative model to H has been formulated.* 

Consider finally the situation in which the subjects are again given rather 
than sampled, but B and B are attributes (for example, male or female, 
smoker or nonsmoker) which cannot be assigned to the subjects at will. 
Then there exists no stochastic basis for answering the question whether 
observed differences in the rates X/M and Y/N correspond to differences 
between B and B, or whether they are accidental. An approach to the 
testing of such hypotheses in a nonstochastic setting has been proposed by 
Freedman and Lane (1982). 

The various models for the 2 X 2 table discussed in Sections 6 and 7 may 
be characterized by indicating which elements are random and which fixed: 


(i) All margins and s random (Poisson). 
(i) All margins are random, s fixed (multinomial sampling). 
(iii) One set of margins random, the other (and then a fortiori s) fixed 
(binomial sampling). 
(v) All margins fixed. Sampling replaced by random assignment of 
subjects to treatments. 
(v) All aspects fixed; no element of randomness. 


In the first three cases there exist UMP unbiased one- and two-sided tests of 
the hypothesis of independence of A and B. These tests are carried out by 
conditioning on the values of all elements in (i)-(iii) that are random, so 
that in the conditional model all margins are fixed. The remaining random- 
ness in the table can be described by any one of the four cell entries; once it 
is known, the others are determined by the margins. The distribution of 
Such an entry under H has the hypergeometric distribution given at the end 
of Section 5. 

The models (i)-(iii) have a common feature. The subjects under observa- 
tion have been obtained by sampling from a population, and the inference 
Corresponding to acceptance or rejection of H refers to that population. 
This is not true in cases (iv) and (v). 


“The one-sided test is of course UMP against the class of alternatives defined by the right 
Side of (21), but no reasonable assumptions have been proposed that would lead to this class. 
For suggestions of a different kind of alternative see Gokhale and Johnson (1978). 
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In (iv) the subjects are given, and a probabilistic basis is created by 
assigning them at random, m to B and n to B. Under the hypothesis H of 
no treatment difference, the four margins are fixed without any condi- 
tioning, and the four cell entries are again determined by any one of them, 
which under H has the same hypergeometric distribution as before. The 
present situation differs from the earlier three in that the inference cannot 
be extended beyond the Subjects at hand.* 

The situation (v) is outside the Scope of this book, since it contains no 
basis for the type of probability calculations considered here. Problems of 
this kind are however of Breat importance, since they arise in many 


observational (as opposed to experimental) studies. For a related discussion, 
see Finch (1979). 


8. SOME THREE-FACTOR CONTINGENCY TABLES 


When an association between A and B exists in a 2 x 2 table, it does not 
follow that one of the factors has a causal influence on the other. Instead, 
the explanation may, for example, lie in the fact that both factors are 
causally affected by a third factor C. If C has K Possible outcomes 
C, +++) Ck, one may then be faced with the apparently paradoxical situa- 
tion that 4 and B are independent under each of the conditions C, 
(Km. Kk ) but exhibit Positive (or negative) association when the tables 
are aggregated over C, that is, 
combined into a single one showin, 
[An interesting example is discussed by Bickel et al. (1977); see also Lindley 
and Novick (1981).] In order to determine whether i 

B in the aggregated table is indeed “spuri x 


table which extends the 2 x 2 table 


of the 4K cellcounts then is multinomi ith 
tnomial with iliti 
Pisco Pac, Pic, for the “yaaa EE 


outcomes indicated by the subscripts. If A, 
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denotes the AB odds ratio for C, defined by 


| Vac Pasc,Pásc, _ Pab\c, Panic, 


, 
Pasc,Pábc, Panic, Pabic, 


where p,5c,... denotes the conditional probability of the indicated event 
given C,, then the hypothesis to be tested is A, =1 for all k. 

A second scheme takes samples of size s, from C, and classifies the 
subjects as AB, AB, AB, or AB. This is the case of K independent 2 x 2 
tables, in which one is dealing with K quadrinomial distributions of the 
kind considered in the preceding sections. Since the kth of these distribu- 
tions is also that of the same four outcomes in the first model conditionally 
given C,, we shall denote the probabilities of these outcomes in the present 
model again by p,gic,... . » 

To motivate the next sampling scheme, suppose that A and A represent 
success or failure of a medical treatment, B and B that the treatment is 
applied or the subject is used as a control, and C, the kth hospital taking 
Part in this study. If samples of size n, and. m, are obtained and are 
assigned to treatment and control respectively, we are dealing with K pairs 
of binomial distributions. Letting Y, and X, denote the number of successes 
obtained by the treatment subjects and controls in the kth hospital, the 
joint distribution of these variables by Section 5 is 


MENIN Pik 
[ri st) ate eo ( outs A, + EG + »)les 7. 


Where p,, and q,,, (p>, and q,,) denote the probabilities of success and 
failure under B (under B). À : 

The above three sampling schemes lead to 2 X 2 x K tables in which 
Tespectively none, one, or two of the margins are fixed. Alternatively, in 
Some situations a model may be appropriate in which the 4K variables 
Xk» Xt, Y,, Y/ are independent Poisson with expectations À , yc, ... . In this 
case, the total sample size s is also random. 

For a test of the hypothesis of conditional independence of 4 and B 
given C, for all k (ie. that Aj = --- =A,=1), see Problem 43 of 
Chapter 8. Here we shall consider the problem under the simplifying 
assumption that the A, have a common value A, so that the hypothesis 
Ieduces to H: A = 1. Applying Theorem 3 to the third model (K pairs of 
binomials) and assuming the alternatives to be A > 1, we see that a UMP 
Unbiased test exists and rejects H when LY, > C(X,  Y,..., Xy + Yp), 
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here C is determined so that the conditional probability of rejection, given 
that x, + Y, = t,, is a for all k = 1,..., K. It follows from Section 5 that 
the conditional joint distribution of the Y, under H is 


Pal Yi 9 y... Yo yp [Xo Y= tj km LX] 


| m, | | 
le — Y J NIk 
IT Mk + ny, 
tk 
The conditional distribution of LY, can now be obtained by adding the 


probabilities over all ( Jr» ---, Jy) whose sum has a given value. Unless the 
numbers are very small, this is impractical and approximations must be 


occurs when Z > 0 and failure when Z < 0. If Z is distributed as Fi (Z= t) 
with location parameter ¢, we have p —1-— F(-f) and Q= F(=) Of 
particular interest is the logistic distribution, for which F(x) =1/(1 + e). 


In this case p = et/(1 + et), 4 7 l/( + ef), and hence log( p/q) = (. 
Applying this fact to the Success probabilities 


foo ca ad dye Pj =T- F(—§,) 


> 


we find that 


9, = log A, = oe [4 a aa Or 
Nk 
so that ,, = 1, + 9,. In this model, H’ thus 
that $5, = fu 6, that is, that the treatment shi 
underlying response by a constant amount [^ 
If it is assumed that F is normal rather than logistic, F(x) = (x) say, 
then ¢ = -X p), and Constancy of $5, — f... requires the much more 
cumbersome condition STGP), pi) = constant. However, the 
functions log( p/q) and p) agree quite well in the range .1 < p < .9 
[see Cox (1970, p. 28)], and the assumption 


F of constant A, in the logistic 
response model is therefore close to the corresponding assumption for an 


4.8] SOME THREE-FACTOR CONTINGENCY TABLES 165 


underlying normal response.* [The so-called loglinear models, which for 
contingency tables correspond to the linear models to be considered in 
Chapter 7 but with a logistic rather than a normal response variable, 
Provide the most widely used approach to contingency tables. See, for 
example, the books by Cox (1970), Haberman (1974), Bishop, Fienberg, and 
Holland (1975), Fienberg (1980), Plackett (1981), and Agresti (1984).] 

The UMP unbiased test, derived above for the case that the B- and 
C-margins are fixed, applies equally when any two margins, any one margin, 
or no margins are fixed, with the understanding that in all cases the test is 
carried out conditionally, given the values of all random margins. 

The test is also used (but no longer UMP unbiased) for testing H : A, = 
‘+> = A, — 1 when the A's are not assumed to be equal but when the 
A, — 1 can be assumed to have the same sign, so that the departure from 
independence is in the same direction for all the 2 x 2 tables. A one- or 
two-sided version is appropriate as the alternatives do or do not specify the 
direction. For a discussion of this test, the Cochran-Mantel-Haenszel test, 
and some of its extensions see the reviews by Landis, Heyman, and Koch 
(1978), Darroch (1981), and Somes and O'Brien (1985). 

Consider now the case K — 2, with m, and n, fixed, and the problem of 
testing H’: A, = A, rather than assuming it. The joint distribution of the 
X's and Y’s given earlier can then be written as 


[& eo 
Pu 


A 
xem salop $ + (Qn + log, + X (x; inp. : 
1 


and H' is rejected in favor of A,> A, if Y, > C, where C depends on 
Y, + Y, X, + Y, and X, + Yj, and is determined so that the conditional 
Probability of rejection given Y, + Y; = w, X, + Y, = t, X;*Y,-t;is 
a. The conditional null distribution of Y, and Y, given X, + Y, — t, 
(k = 1,2), by (21) with A in place of p is 


seta, s (ms san m 


and hence the conditional distribution of Y,, given in addition that Y, + Y, 


*The problem of discriminating between a logistic and normal response model is discussed 
by Chambers and Cox (1967). 
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= w, is of the form 


kt aw iis ni come presa il d 


Some approximations to the critical value of this test are discussed by Birch 
(1964); see also Venable and Bhapkar (1978). [Optimum large-sample tests 
of some other hypotheses in 2 X 2 X 2 tables are obtained by Cohen, 
Gatsonis, and Marden (1983).] 


9. THE SIGN TEST 


To test consumer preferences between two products, a sample of n subjects 
are asked to state their preferences. Each subject is recorded as plus or 
minus as it favors product B or A. The total number Y of plus signs is then 
a binomial variable with distribution b(p, n). Consider the problem of 
testing the hypothesis p = 3 of no difference against the alternatives p # 1. 
(As in previous such problems, we disregard here that in case of rejection it 
will be necessary to decide which of the two products is preferred.) The 
appropriate test is the two-sided sign test, which rejects when IY — in| is 
too large. This is UMP unbiased (Section 2). 

Sometimes the subjects are also given the possibility of declaring them- 
selves as undecided. If p , p,, and p, denote the probabilities of prefer- 
ence for product A, product B, and of no preference respectively, the 
numbers X, Y, and Z of decisions in favor of these three possibilities are 
distributed according to the multinomial distribution 


n! 
(22) yiyi PtP (xt+y+z=n), 


and the hypothesis to be tested is H : Ps 
also be written as 


n! B4 »y P z 
23 ——|—————]|[——25-— 
e» ee) eae O~po~p.)", 


= p... The distribution (22) can 


and is then seen to constitute an exponential family with U — Y, T — Z, 
6 = loglp./(1 — po = p.) 9 = log py/(. — po — p ,)) Rewriting the hy- 
pothesis H as p,= 1 — p, — p,, itis seen to be equivalent to 0 = 0. There 
exists therefore a UMP unbiased test of H, which is obtained by considering 
z as fixed and determining the best unbiased conditional test of H given 
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Z= z. Since the conditional distribution of Y given z is a binomial 
distribution b( p, n — z) with p = P./(p.+ p_), the problem reduces to 
that of testing the hypothesis p = } in a binomial distribution with n — z 
trials, for which the rejection region is |Y — 1(n — z)| > C(z). The UMP 
unbiased test is therefore obtained by disregarding the number of cases in 
which no preference is expressed (the number of ties), and applying the sign 
test to the remaining data. 

The power of the test depends strongly on Po, Which governs the 
distribution of Z. For large po, the number n — z of trials in the conditional 
binomial distribution can be expected to be small, and the test will thus 
have little power. This may be an advantage in the present case, since a 
sufficiently high value of po, regardless of the value of P./p_, implies that 
the population as a whole is largely indifferent with respect to the products. 

The above conditional sign test applies to any situation in which the 
observations are the result of n independent trials, each of which is either a 
Success (+), a failure (—), or a tie. As an alternative treatment of ties, it is 
Sometimes proposed to assign each tie at random (with probability } each) 
to either plus or minus. The total number Y’ of plus signs after the ties have 
been broken is then a binomial variable with distribution b(7, n), where 
7 = p,+ lp, The hypothesis H becomes 7 = i, and is rejected when 
|Y" — 1n| > C, where the probability of rejection is a when 7 = 1. This test 
can be viewed also as a randomized test based on X, Y, and Z, and it is 
unbiased for testing H in its original form, since p, is = or # p as is 
= or +}. Since the test involves randomization other than on the 
boundaries of the rejection region, it is less powerful than the UMP 
unbiased test for this situation, so that the random breaking of ties results in 
a loss of power. 

This remark might be thought to throw some light on the question of 
whether in the determination of consumer preferences it is better to permit 
the subject to remain undecided or to force an expression of preference. 
However, here the assumption of a completely random assignment in case 
of a tie does not apply. Even when the subject is not conscious of a definite 
Preference, there will usually be a slight inclination toward one of the two 
Possibilities, which in a majority of the cases will be brought out by a forced 
decision. This will be balanced in part by the fact that such forced decisions 
are more variable than those reached voluntarily. Which of these two factors 
dominates depends on the strength of the preference. 

Frequently, the question of preference arises between a standard product 
and a possible modification or a new product. If each subject is required to 
express a definité preference, the hypothesis of interest is usually the 
One-sided hypothesis p,<p_, where + denotes a preference for 
the modification. However, if an expression of indifference is permitted, the 
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hypothesis to be tested is not p,« p. but rather PsS Pot p, since 
typically the modification is of interest only if it is actually preferred. As 
was shown in Chapter 3, Example 8, the one-sided sign test which rejects 
when the number of plus signs is too large is UMP for this problem. 

In some investigations, the subject is asked not only to express a 
preference but to give a more detailed evaluation, such as a score on some 
numerical scale. Depending on the situation, the hypothesis can then take 
on one of two forms. One may be interested in the hypothesis that there is 
no difference in the consumer’s reaction to the two products. Formally, this 
states that the distribution of the scores X... X, expressing the degree of 
preference of the n subjects for the modified product is symmetric about the 
origin. This problem, for which a UMP unbiased test does not exist without 
further assumptions, will be considered in Chapter 6, Section 10. 

Alternatively, the hypothesis of interest may continue to be H: p,=p_. 
Since p_= P( X < 0) and P.= P(X > 0), this now becomes 


H:P(X»0) = P(X <0}. 


fact that the set of its possible values is given, the sign test based on the 


number of X's that are Positive and negative continues to be UMP 
unbiased. 


To see this, note that any distribution of X can be specified by the 
probabilities 
poet PUES Dar Pt Sed inops e P(X - 0), 
: à + of X given X < 0 and X > 0 
respectively. Consider any fixed distributions F^, F^. and denote by Fo 
-= Fl, F,— F} and arbitrary 


A 7 +, F4. Let $ be any other unbiased 
test of H in .£, and consider any fixed alternative, which without loss of 


umed to be in Fo. Since ¢ is unbiased for F, it is 
c Power of à, against the particular 
alternative is therefore at least as good as that of $. Hence Fai UMP 


To determine the UMP unbiased test of H ig. 


; » let the densities of F’ 
, 0: us 
and F; with respect to some measure I: be f* and ft. The joint density of 
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the X’s at a point (x,,..., x,) with 


Rayo dens Rye Dm emn ty ine Nps wes Xi 


P. popa H7 (xi)... S Gu )F4(%4,)--- f(x, ). 


The set of statistics (7, s, m) is sufficient for ( p. , po, p+), and its distribu- 
tion is given by (22) with x =r, y =m, z = s. The sign test is therefore 
seen to be UMP unbiased as before. 

A different application of the sign test arises in the context of a 2 X 2 
table for matched pairs. In Section 5, success probabilities for two treat- 
ments were compared on the basis of two independent random samples. 
Unless the population of subjects from which these samples are drawn is 
fairly homogeneous, a more powerful test can often be obtained by using a 
sample of matched pairs (for example, twins or the same subject given the 
treatments at different times). For each pair there are then four possible 
outcomes: (0, 0), (0, 1), (1, 0), and (1, 1), where 1 and 0 stand for success and 
failure, and the first and second number in each pair of responses refer to 
the subject receiving treatment 1 or 2 respectively. 

The results of such a study are sometimes displayed in a 2 X 2 table, 


Which despite the formal similarity differs from that considered in Section 6. 
If a sample of s pairs is drawn, the joint distribution of X, Y, X’, Y' as 
before is multinomial, with probabilities Poo» Poi Pio» P: The success 
Probabilities of the two treatments are 7, = Pio + Pu for the first and 
7; = po, + py, for the second treatment, and the hypothesis to be tested is 

: 7, = m, Or equivalently pio = Po rather than p;9o; = PooP11 aS it was 
earlier, 

In exponential form, the joint distribution can be written as 


s!pi Pa Pio Poo 
24 pacer IAE — +(x’ + y)log— + x log— |. 
(24) xlx'lyly’! 2 dg vas Pu Be 


There exists a UMP unbiased test, McNemar’s test, which rejects H in 
favor of the alternatives Pio < Po, when Y > C(X' + Y, X), where the 
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conditional probability of rejection given X’ + Y = d and X= x isa for 
all d and x. Under this condition, the numbers of pairs (0,0) and (1, 1) are 
fixed, and the only remaining variables are Y and Y'—q— Y which 
specify the division of the d cases with mixed response between the 
outcomes (0, 1) and (1,0). Conditionally, one is dealing with d binomial 
trials with success probability p — PoV/ÁPoi + Pio), H becomes p = j, and 
the UMP unbiased test reduces to the sign test. [The issue of conditional 
versus unconditional power for this test is discussed by Frisén (1980).] 

The situation is completely analogous to that of the sign test in the 
presence of undecided opinions, with the only difference that there are now 
two types of ties, (0,0) and (1,1), both of which are disregarded in 
performing the test. 


10. PROBLEMS 
Section 1 


l. Admissibility, Any UMP unbiased test $ is admissible in the sense that 
there cannot exist another test $ı which is at least as powerful as $, against 
all alternatives and more powerful against some. 

[If $ is unbiased and 9' is uniformly at least as powerful as $, then ¢’ is also 
unbiased.] 


p p-values. Consider a family of tests of H: 9 = b (or 8 < &), with level-a 
rejection regions S, such that (a) Pa (XeS,) = a for all 0 < a < 1, and (b) 
S7 Na>apSa for all 0 < % < 1, which in particular implies S, C Sy for 
& « a. Š j 

(i) Then the p-value & is given by & = &(x) = inff a: x € S,}. 
(ii) When 9 = 6, the distribution of & is the uniform distribution over (0, 1). 


(ui) If ke tests S, are unbiased, the distribution of â under any alternative 6 
satisfies 


P, (à <a} 2% {&<a} =a, 


So that it is shifted toward the origin, 


lependent experiments, they can 
of the hypothesis. 


F&(X € S.) = B,(6), 
2 a if Ø is an alternative to 4] bin hal) 


*For discussions of such tests see for exam i 
ple Koziol and Perlman (1978 ; Berk and Coh: 
pei nai and George (1979), Scholz (1982), and the related work ^ Mac (1989), 
Ssociated confidence intervals are Proposed by Littell and Louv (1981). i 
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Section 2 

Let X have the binomial distribution b(p, n), and consider the hypothesis 
H:p = po at level of significance a. Determine the boundary values of the 
UMP unbiased test for n = 10 with a = .1, Po = .2 and with a = .05, p, = 4, 
and in each case graph the power functions of both the unbiased and the 
equal-tails test. 
Let X have the Poisson distribution P(r), and consider the hypothesis 
H 1 = 1. Then condition (6) reduces to 

€i-1 gel C,-1 


2 
o % = 
e+ } 1 -¥)——-e = 1 =a, 
NX s c -3i 


provided C, » 1. 


Let 7,/8 have a x?-distribution with n degrees of freedom. For testing 
H: 0 = 1 at level of significance a = .05, find n so large that the power of the 
UMP unbiased test is > .9 against both 0 > 2 and 0 < 1. How large does n 
have to be if the test is not required to be unbiased? 


Let X and Y be independently distributed according to one-parameter ex- 
ponential families, so that their joint distribution is given by 


dP, o, ( x, y) = C(6,) eh? du (x) K(6,) e^» dv( y). 


Suppose that with probability 1 the statistics T and U each take on at least 
three values and that (a, b) is an interior point of the natural parameter space. 
Then a UMP unbiased test does not exist for testing H:6; =a, 0, = b 
against the alternatives 0, # a or 0, * b.* 

[The most powerful anbiased tests against the alternatives 0, * a, 0, = b and 
9, = a, 6, + b have acceptance regions C, < T(x) < C, and K, < U(y) < 
K, respectively. These tests are also unbiased against the wider class of 
alternatives K: 6, # a or 6, # b or both] 


Let ( X, Y) be distributed according to the exponential family 


dP}, o (x, y) = C(O, 6;) e%**% d(x, y). 


The only unbiased test for testing H:0, < a, 0, < b against K: 0, >a or 
9, > b or both is $(x, y) = a. 


“For counterexamples when the conditions of the problem are not satisfied, see Kallenberg 


(1984), 
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[Take a = b = 0, and let (6, 0,) be the power function of any level-a test. 
Unbiasedness implies 8(0,0,) = a for 6, < 0 and hence for all 0, since 
B(0, 07) is an analytic function of 0,. For fixed 6, > 0, B(0,, 02) considered as 
a function of 6, therefore has a minimum at 6, = 0, so that OB(8,, 0,)/30, 
vanishes at 0, = 0 for all positive @,, and hence for all 6,. By considering 
alternatively positive and negative values of @, and using the fact that the 
partial derivatives of all orders of B(0,, 6,) with respect to 6, are analytic, one 
finds that for each fixed @, these derivatives all vanish at 0, = 0 and hence 
that the function £ must be a constant. Because of the completeness of ( X, Y ), 
B(0,, 02) = a implies $(x, y) = a.) 


For testing the hypothesis H: 0 — 6, (6, an interior point of Q) in the 
one-parameter exponential family of Section 2, let € be the totality of tests 
satisfying (3) and (5) for some — co < C, <O < œ and0 <y, y « 1. 


() @ is complete in the sense that given any level-a test à; of H there 
exists $ € € such that ¢ is uniformly at least as powerful as $o- 


(ü) If $,, $, € €, then neither of the two tests is uniformly more powerful 
than the other. 


(iii) Let the problem be considered as a two-decision problem, with decisions 
dy and d, corresponding to acceptance and rejection of H, and with loss 
function L(0, d;) = L,(0), i = 0,1. Then € is minimal essentially com- 
plete provided L,(0) < Lo(0) for all 6 # 6,. 

(iv) Extend the result of part (iii) to the hypothesis H’ : 0, <0 <0. 


[G): Let the derivative of the power function of $ at 6 be f; (0) = p. Then 
there exists $ € € such that B/(8) =p and $ is UMP among all tests 
satisfying this condition. 

(ii): See Chapter 3, end of Section 7. 

(iii): See Chapter 3, proof of Theorem 3.] 


Section 3 


Let X,,..., X, be a sample from (i) the normal distribution 2), with a 
fixed and 0<oa 06; (ii) the uniform distribution U(0 — CTAN E- 9 
< 00; (iii) the uniform distribution U(0,,0,), — œ < 0, «0, < 3 "For these 
Ex ur pA distributions the following statistics are sufficient: nre 
be: ); (i) and (ii), T = (min( X,,..., Xa), max(X,,..., X. )). The family 
stributions of T is complete for case (iii), but for (i) and (ii) it is not 
complete or even boundedly complete. 
[@): The distribution of EX/ ext does not depend on o.) 


"8 eX ird des Vrae samples from N(E, o*) and NE, 72), Then 
ORNA MV : of ay in Example 5 was seen not to be complete, is 


[Let f(t) be 1 or —1 as y — X is positive or not.] 


410] 
11. 
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Counterexample. Let X be a random variable taking on the values 
-1,0,1,2,... with probabilities 


h(X--1)-6  B(X-x)-(-0Y6*, x=0,1,.... 


Then P = { P}, 0 < 0 < 1} is boundedly complete but not complete. 


The completeness of the order statistics in Example 6 remains true if the family 
F is replaced by the family F, of all continuous distributions. 

[To show that for any integrable symmetric function $, fo(%,..., 
x,,) dF(xi)... dF(x,) = 0 for all continuous F implies $ = 0 a.e., replace F 
by a, F; + +++ *a,F,, where 0 < a, < 1, Ea, = 1. By considering the left side 
of the resulting identity as a polynomial in the a’s one sees that 
J$(%,,..., x,) dF (x,)... dF,(x,) = 0 for all continuous F.. This last equa- 
tion remains valid if the F, are replaced by I, (x) F(x), where T,,(x) = 1 if 
X <a, and = 0 otherwise. This implies that ¢ = 0 except on a set which has 
measure 0 under F X --- XF for all continuous F.) 


Determine whether T is complete for each of the following situations: 


G) X,,..., X, are independently distributed according to the uniform distri- 
bution over the integers 1,2,..., 0 and T = max(X;,..., X,). 

(i) X takes on the values 1,2,3,4 with probabilities pg, p^q, pq?,1 — 2pq 
respectively, and T = X. 

Section 4 

Measurability of tests of Theorer: 3. The function $, defined by (16) and (17) 

is jointly measurable in u and t. ; j 

[With C, = v and C, = w, the determining equations for v, w, Yı, y; are 

(25) E(v -) + [1 - £(v)] + n[ R(x) - E» -)] 


+n[ E(w) - £(v -)] =a 


and 

Q6) G(v -) + [1 - G(»)] + n[GCo) - GC -)] 
+n[G(w) - G(»-)] = a, 

where 


(27) F(u) -f G(0,) e^» dr,(y), Gu) = f. G(&)e^ an) 


denote the conditional cumulative distribution function of U given t when 
9 = 0, and 6 = 6, respectively. 
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(1) Foreach0 < y < a let v(y, t) = F (y) and w(y, t) = R41 — a + y), 
where the inverse function is defined as in the proof of Theorem 3. Define 
m» t) and y2(y, t) so that for v = v( y, t) and w = w( y, t), 


F(» —) + w[£(v) - E(v -)] =y, 
1- E(w) + y[£(w) - £(w-)] 2 a - y. 


(2) Let H(y, t) denote the left-hand side of (26), with v — v( y, 1), etc. Then 
H(0, t) > a and H(a, t) < a. This follows by Theorem 2 of Chapter 3 from 
the fact that v(0, 7) = —oo and w(a, t) = oo (which shows the conditional 
tests corresponding to y — 0 and y — a to be one-sided), and that the 
left-hand side of (26) for any y is the power of this conditional test. 

(3) For fixed t, the functions 


A,(y,t) = G(v-) + nlG,(v) = G(v -)] 


and 
H(y,t) =1— G(w) + v[G(w) - G(w -)] 


are continuous functions of y. This is a consequence of the fact, which follows 
from (27), that ae. PT the discontinuities and flat stretches of F and G, 
coincide. 

(4) The function H( Y, t) is jointly measurable in y and t. This follows from 


the continuity of H by an argument similar to the proof of measurability of 
F(u) in the text. Define 


y(t) = int( y: H(y,t) < a), 


and let v(t) = v[ y(t), t], etc. Then (25) and (26) are satisfied for all t. The 
measurability of v(t), w(t), v(t), and y,(t) defined in this manner will follow 


from measurability in 1 of y(t) and E-'[y(r)]. This i th 
relations, which hold for all real c, cR: peer! tbe 


fax) <c} = YU (6 H(r,t) <a}, 


where r indicates a rational, and 


GE D()] <c} = C: (1) - E(9) «0)] 


Continuation. The function 


% defined b: is joi 
measurable in u and z. 1 5 TA SAOD aand C äisjointly 
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16. 


17. 


18. 


[The proof, which otherwise is essentially like that outlined in the preceding 
problem, requires the measurability in z and 1 of the integral 


g(z.0) = f^ uaE(u). 


This integral is absolutely convergent for all 1, since F is a distribution 
belonging to an exponential family. For any z < oo, g(z, t) = lim 8, (2, t), 
where 


8,(Z,%) = iR (:- male i -0) =F 2- E -9. 


j=1 


and the measurability of g follows from that of the functions g,. The 
inequalities corresponding to those obtained in step (2) of the preceding 
problem result from the property of the conditional one-sided tests established 
in Problem 22 of Chapter 3.] 


The UMP unbiased tests of the hypotheses H;,..., H, of Theorem 3 are 
unique if attention is restricted to tests depending: on U and the T's. 


Section 5 


Let X and Y be independently distributed with Poisson distributions P(A) 
and P(,). Find the power of the UMP unbiased test of H: p < A, against the 
alternatives À = 1, p = 2; À = 1, p = 2; À = 10, p = 20; À = .1, p = 4; at 
level of significance a = .1. 

[Since T = X + Y has the Poisson distribution P(A + p), the power is 


B- £ po eon, 


t=0 


where B(t) is the power of the conditional test given 1 against the alternative 
in question.] 

Sequential comparison of two binomials. Consider two sequences of binomial 
trials with probabilities of success p, and p; respectively, and let p — 
(22/42) + (1/4). 


(i) If a< B, no test with fixed numbers of trials m and n for testing 
H: p = p, can have power > f against all alternatives with p = py. 

(i) The following is a simple sequential sampling scheme leading to the 
desired result. Let the trials be performed in pairs of one of each kind, 
and restrict attention to those pairs in which one of the trials is a success 
and the other a failure. If experimentation is continued until N, such 
pairs have been observed, the number of pairs in which the successful 
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19. 


20. 


Anderson and Goodman (1957), Goodman (1958), Billingsley (1961), 
) ^ 


trial belonged to the first series has the binomial distribution b(v, N) 
with 7 = piq;/(p,q; + p24) 7 1/(1 + p). A test of arbitrarily high 
power against p, is therefore obtained by taking N large enough. 


(ii) If p/p = À, use inverse binomial sampling to devise a test of H: À = Xo 
against K: A > Xs. 


x, y.* For the case that (X, Y) takes on the four pairs of values (0,0), (0, 1), 


Property? 
P{X, = 1,34) = P{X = 1]x,_,} 
and the property of Stationarity according to which P{X, = 1) and P(X, = 


1|x,_,} are independent of i. The distribution of the X’s is then specified by 
the probabilities 


Pm P(X 71X-,21) and py = P(X - 1x, , = 0) 
and by the initial probabilities 
7, = P{X,=1} and % = 1-9, = P{ X, =0}. 
(i) Stationarity implies that 


Po qı 


™ = ™ = j 
Potq' 5 Dota 


e ENSE ca 
p, q, PA NG *, 


ice in these and more general Markov chains is discussed, for example, in 
Denny and Wright 
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21. 


where m and n denote the numbers of zeros and ones, and u and v the 
numbers of runs of zeros and ones in the sequence. 


Continuation. For testing the hypothesis of independence of the X’s, H : Po 
= pı, against the alternatives K : pọ < p}, consider the run test, which rejects 
H when the total number of runs R = U + V is less than a constant C(m) 
depending on the number m of zeros in the sequence. When R = C(m), the 
hypothesis is rejected with probability y(m), where C and y are determined 
by 


Pu {R< C(m)|m) + y(m)Py {R= C(m)|m) = a. 


(Gi) Against any alternative of K the most powerful similar test (which is at 
least as powerful as the most powerful unbiased test) coincides with the 
run test in that it rejects H when R < C(m). Only the supplementary 
rule for bringing the conditional probability of rejection (given m) up to 
a depends on the specific alternative under consideration. 

(ii) The run test is unbiased against the alternatives K. 

(iii) The conditional distribution of R given m, when H is true, is* 


hd pulsar 
m-1\(n-1 m-1)\(n-1 
P{R=2r+1}= PiN Di ye 1 


[G): Unbiasedness implies that the conditional probability of rejection given m 
is a for all m. The most powerful conditional level-a test rejects H for those 


sample sequences for which A(u, v) = ( po/p1)'(q/40)" is too large. Since 
Po < p, and q, < qo and since |v — u| can only take on the values 0 and 1, it 


follows that 
A(1,1) > A(1,2),  A(2,1) > A(2,2) > A(2,3),  4(32)» ---. 


Thus only the relation between A(i,i +1) and A(i + 1, i) depends on the 
specific alternative, and this establishes the desired result. y 
(ii): That the above conditional test is unbiased for each m is seen by writing 
its power as 


B(po, pim) = (1 — y) P(R < C(m)|m} + yP{R s C(m)|m}, 


“This distribution is tabled by Swed and Eisenhart (1943) and can be obtained from the 


hypergeometric distribution [Guenther (1978)]. For further discussion of the run test, see 
Wolfowitz (1943), 
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since by (i) the rejection regions R < C(m) and R< C(m) +1 are both 
UMP at their respective conditional levels. 

(iii): When H is true, the conditional probability given m of any set of m 
zeros and n ones is 1/( ™” a m The number of ways of dividing n ones into r 


oups is (” -1 , and that of dividin, m zeros into r + 1 groups is (” - 1. 
groups is [^ B ; 


The conditional probability of getting r + 1 runs of zeros and r runs of ones is 


therefore 
(zzi) 


("a") 


r + 1 runs of ones.] 


() Based on the conditional distribution of X,,..., X, given X, = x, in the 
model of Problem 20, there exists a UMP unbiased test of H: py =p, 
against p, > py for every a. 

(i) For the same testing problem, without conditioning on X, there exists a 
UMP unbiased test if the initial Probability 7, is assumed to be com- 


pletely unknown instead of being given by the value stated in (i) of 
Problem 20. 


[The conditional distribution of X,,..., X, given X, is of the form 


CCo Po» Pis dos qi) pip}? iq hy, yy, 21, 2), 


where y, is the number of times a 1 follows a 1, yy the number of times a 1 


He a 0, and so on, in the Sequence x), X5,.... X,. [See Billingsley (1961, 


Rank-sum test. Let Y, Yy be ind istri i 
OTEREN Ror, ependently distributed according to the 
binomial distributions b(pi, nj), i9 1,..., N, vien Cesena 
1 


[mE LE 
Pi 1 + e (98x) 


4.10] PROBLEMS 179 


24. 


25. 


26. 


27. 


(i) Suppose in particular that x; = Ai, where A is known, and that n; = 1 
for all i. Let n be the number of successes in the N trials, and let these 
successes occur in the sst, s, nd,..., s,th trial, where s, < s < --- < 
s,. Then the UMP unbiased test for testing H: 8 — 0 against the 
alternatives B > 0 is carried out conditionally, given n, and rejects when 
the rank sum Y7..,s, is too large. 

(iii) Let Y,,..., Y, and Z,,..., Zy be two independent sets of experiments 
of the type described at the beginning of the problem, corresponding, 
say, to two different drugs. If Y, is distributed as b( p;, m;) and Z; as 
b(m, nj), with 


1 1 
Pi = 1 + e (9*84)* 9n" 14e O*89)* 


then UMP unbiased tests exist for the four hypotheses concerning y — a 
and 8 — f. 


Section 8 
Ina2x2x2 table with m, =3, nj =4; m; —4, n; — 4; and h — 3, 
t{ = 4, t, = t} = 4, determine the probabilities that P(Y, + Y; < k|X, + Y, = 
tj, i = 1,2) for k = 0,1,2,3. 
Ina2X2XxX K table with A, =A, the test derived in the text as UMP 
unbiased for the case that the B and C margins are fixed has the same 
property when any two, one, or no margins are fixed. 


Let X; (i, j,k = 0,1, L= 1,..., L) denote the entriesina2X 2X2X L 


i 


table with factors A, B, C, and D, and let 


T, Paico, Pisco; PABCD, PABCD, 
FALBGDIEM BGDEIEHC hat ehh 

t = 73. x 72A 
Pancp,DABcD, PABCD, PABCD, 


Then 


(i) under the assumption T, =T there exists a UMP unbiased test of the 
hypothesis T < T, for any fixed I; ; . 
Gi) When / = 2, there exists a UMP unbiased test of the hypothesis T, = T, 


—in both cases regardless of whether 0, 1, 2 or 3 of the sets of margins are 
fixed. 


Section 9 


In the 2 x 2 table for matched pairs, show by formal computation that a 
conditional distribution of Y given X’ + Y = d and X = x is binomial wi! 
the indicated p. 


180 
28. 


29, 


30. 


31. 
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Consider the comparison of two success probabilities in (a) the two-binomial 
situation of Section 5 with m = n, and (b) the matched-pairs situation of 
Section 9. Suppose the matching is completely at random, that is, a random 
sample of 2n subjects, obtained from a population of size N (2n < N), is 
divided at random into n pairs, and the two treatments B and B are assigned 
at random within each pair. 


(i) The UMP unbiased test for design (a) (Fisher’s exact test) is always more 
powerful than the UMP unbiased test for design (b) (McNemar’s test). 

(ii) Let X, (respectively Y,) be 1 or 0 as the Ist (respectively 2nd) member of 
the ith pair is a success or failure. Then the correlation coefficient of X, 
and Y, can be positive or negative and tends to zero as N — oo. 


(Gi): Assume that the kth member of the population has probability of success 
PY under treatment A and Pf? under A} 


In the 2 X 2 table for matched pairs, in the notation of Section 9, the 
correlation between the Tesponses of the two members of a pair is 


us Pu- nm 


en yn - m)o(- m) 


For any given values of m < m, the power of the one-sided McNemar test of 
H: m, = m, is an increasing function of p. 

[The conditional power of the test given X + y = d, X — x is an increasing 
function p = Po/(Poi + p19)] 


Note. The correlation p increases with the effectiveness of the matching, and 


McNemar's test under (b) of Problem 28 soon becomes more powerful than 
Fisher's test under (a). For detailed 


i numerical comparisons see Wacholder and 
Weinberg (1982) and the references given there, 


Additional Problems 
Let X,Y be independent binomial b(p, 
termine whether ( Y, Y) is complete when 


(i) m=n=1, 
(ii) m-2,n-]. 


m) and b(p?, n) respectively. De- 


Let X... X, bea sample from the 
1,..., 0, and let a be à positive integer. 


(i) The sufficient Statistic X i ’ 
Q - (6:9 s C Xn) is complete when the parameter space is 


uniform distribution Over the integers 


(ii) Show that Xen) is not complete when Q = 


0:0 
complete sufficient sanat Grae { 24), a > 2, and find a 
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32. Negative binomial. Let X, Y be independently distributed according to nega- 
tive binomial distributions Nb( p, m) and Nb(p;, n) respectively, and let 
4-71-p. 

(i) There exists a UMP unbiased test for testing H:0 = 9,/q, < &, and 
hence in particular H’ : p; € p;. 

(i) Determine the conditional distribution required for testing H^ when 
m=n=1. 

33. Let X, (i = 1,2) be independently distributed according to distributions from 


the exponential families (12) of Chapter 3 with C, Q, T, and h replaced by C;, 
Q,, T,, and h,. Then there exists a UMP unbiased test of 


G) H:Q,(8,) — Q,(0,) < c and hence in particular of Q5(0,) < Q,(0,); 
Gi) H:Q;(6, + Q,(8,) < c. 


34. Let X, Y, Z be independent Poisson variables with means À, w, v. Then there 
exists a UMP unbiased test of H: Ap < v?. 


35. Random sample size. Let N be a random variable with a power-series 
distribution 


a(n)" 


—— =0,1,... À > 0, unknown). 
may ( ) 


P(N=n)= 


When N =n, a sample X,,..., X, from the exponential family (12) of 
Chapter 3 is observed. On the basis of (N, X,,..., Xy) there exists a UMP 
unbiased test of H: Q(0) < c. 

36. The UMP unbiased test of H: A = 1 derived in Section 8 for the case that the 
B- and C-margins are fixed (where the conditioning now extends to all random 
margins) is also UMP unbiased when 


(G) only one of the margins is fixed; 
(i) the entries in the 4K cells are independent Poisson variables with means 
pcs... and A is replaced by the corresponding cross-ratio of the A's. 
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CHAPTER 5 


Unbiasedness: Applications 
to Normal Distributions; 
Confidence Intervals 


1. STATISTICS INDEPENDENT OF A SUFFICIENT 
STATISTIC 


A general expression for the UMP unbiased tests of the hypotheses H; : 0 < 
Q and H,: 8 = 6, in the exponential family 


(1) dPa a(x) = C(O, exp[OU(x) + Y67,(x)] au(x) 


was given in Theorem 3 of the preceding chapter. However, this turns out to 
be inconvenient in the applications to normal and certain other families of 
continuous distributions, with which we shall be concerned in the present 
chapter. In these applications, the tests can be given a more convenient 
form, in which they no longer appear as conditional tests in terms of U 
given t, but are expressed unconditionally in terms of a single test statistic. 
The following are three general methods of achieving this. 


K three, and has the additional advantage that it derives the 
ks WE of verifying a guessed solution as is the case with methods (ii) 
and (iii), 


(ii) The conditional descriptions (12), (14), and (16) of Chapter 4 can be 
replaced by equivalent unconditional ones, and it is then enough to find an 
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unbiased test which has the indicated structure. This approach is discussed 
in Pratt (1962). 

(iii) Finally, it is often possible to show the equivalence of the test given 
by Theorem 3 of Chapter 4 to a test suspected to be optimal, by means of 
Theorem 2 below. This is the course we shall follow here; the alternative 
derivation (i) will be discussed in Chapter 6. 


The reduction by method (iii) depends on the existence of a statistic 
V = h(U, T), which is independent of T when 0 = 4, and which for each 
fixed ¢ is monotone in U for H, and linear in U for H,. The critical function 
$, for testing H, then satisfies 


1 when v» C, 
(2) $(v) {Yo when v= G, 
0 when v «C, 


where C, and yọ are no longer dependent on t, and are determined by 
(3) Ergi (V) = a. 
Similarly the test 6, of H, reduces to 


1 when v<C, or v> Q, 
(4) o(v) =(y, when v=C, i=1,2, 
0 when Cj, «v «C, 


Where the C's and y's are determined by 


P E, [e ()] = 4 
and 
(9) Es, [V4 (V)] = aE,,(V). 


The corresponding reduction for the hypotheses Hy: 6x0, or 02 6, 
and H,:6, < 0 < 0, requires that V be monotone in U for each fixed t, 
and be independent of T when 6 = 0, and 6 = 6,. The test 4; is then given 
by (4) with the C's and y’s determined by 


(7) Ej; (V) = E,o3(V) =a. 


190 UNBIASEDNESS: APPLICATIONS; CONFIDENCE INTERVALS [5.1 
The test for H, as before has the critical function 


$,(0; a) 21— $3(v;1 — a). 


This is summarized in the following theorem. 


Theorem 1. Suppose that the distribution of X is given by (1) and that 
V = A(U, T) is independent of T when 0 = 9%. Then $, is UMP unbiased for 
testing H, provided the function h is increasing in u for each t, and $, is UMP 
unbiased for H, provided 


h(u,t) 2 a(t)u- b(t) with a(t) » 0. 


The tests $, and $, are UMP unbiased for H, and H; if V is independent of T 
when 0 = 0, and 8,, and if h is increasing in u for each t. 


Proof. The test of H, defined by (12) and (13) of Chapter 4 is equiv- 
alent to that given by (2), with the constants determined by 


Pol V > GG) + volt) P (Y = Cy(z)It} = a. 


By assumption, V is independent of T when 6 = O, and C, and y, 
therefore do not depend on 1. This completes the proof for H;, and that for 
H, and H, is quite analogous. 

The test of H, given in Section 4 of Chapter 4 is equivalent to that 
dene by (4) with the constants C, and y, determined by Eg [$4(V, t)|t] ^ a 
an 


sue, t) E | = en HO | 


Which reduces to 


Ey,[Vo4(V, t)|¢] = aE, [V|r]. 


Since V is independent of T for 0 = 6, 
proved. 


To prove the required independence of V and T in applications of 
Theorem 1 to Special cases, the standard methods of 


» SO are the C's and y's as was to be 
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Theorem 2. (Basu). Let the family of possible distributions of X be 
P= (P5, 9 € w}, let T be sufficient for P, and suppose that the family PT 
of distributions of T is boundedly complete. If V is any statistic whose 
distribution does not depend on 9, then V is independent of T. 


Proof. For any critical function $, the expectation Eyp(V) is by 
assumption independent of 2. It therefore follows from Theorem 2 of 
Chapter 4 that E[¢(V)|t] is constant (a.e. P7) for every critical function $, 
and hence that V is independent of T. 

For converse aspects of this theorem see Basu (1958) Koehn and 
Thomas (1975), Bahadur (1979), and Lehmann (1980). 


Corollary 1. Let P be the exponential family obtained from (1) by letting 
0 have some fixed value. Then a statistic V is independent of T for all 9 
provided the distribution of V does not depend on 9. 


Proof. It follows from Theorem 1 of Chapter 4 that 2^ is complete 
and hence boundedly complete, and the preceding theorem is therefore 
applicable. 

Example 1. Let X,,..., X, be independently, normally distributed with mean £ 
and variance c?. Suppose first that o? is fixed at oj. Then the assumptions of 
Corollary 1 hold with T = X — XX,/n and $ proportional to $. Let f be any 
function satisfying 


P(X 6... x +E) m f(x) for all real c. 


V 2f(38 Xn) 


then also V = f(X, — £,..., X, — £). Since the variables X, — £ are distributed as 
N(0, o2), which does not involve £, the distribution of V does not depend on £. It 
follows from Corollary 1 that any such statistic V, and therefore in particular 
V — X(X, — X)’, is independent of X. This is true for all c. DUE 

Suppose, on the other hand, that £ is fixed at £y. Then Corollary 1 applies with 
T — Y(X, — £y)? and 9 = —1/26?. Let f be any function such that 


f(c,...,ex,) = f... x.) for all p 
and let 
v=f(% -6eeoX bs $o). 


Then V is unchanged if each X, — £y is replaced by (X, — £9)/6, and since these 
variables are seta distributed with zero mean and unit variance, the distribution 
of V does not depend on oc. It follows that all such statistics V, and hence for 
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example 
X —£o X ip 


jE«x- tU VEX =i) 


are independent of Y( X, — £)”. This, however, does not hold for all £, but only 
when £ = £,. 


Example 2. Let U,/o; and U,/oj be independently distributed according to 
X2-distributions with f, and f; degrees of freedom respectively, and suppose that 
9j /oj = a. The joint density of the U's is then 


1 
Cul) yr) m * w) 
9 


so that Corollary 1 is applicable with T = aU, + U, and $= —1/202. Since the 
distribution of 


does not depend on o, V is independent of aU, + U,. For the particular case that 
9; = 0, this proves the independence of U,/U, and U, + U;. 


Example 3. Let (X,,..., X,) and (Y,,..., ¥,) be samples from normal distribu- 
tions N(£, o?) and N(n, 7?) respectively. Then T = (X, Y. X2. Y, LY?) is sufficient 
for (£, 07, n, 7?) and the family of distributions of T is complete. Since 


E-I- ¥) 


VEX - X» EG Fr 


is unchanged when X, and Y, are replaced by (X, — §)/o and (Y, — n)/r, the 


distribution of V does not depend on any of the parameters, and Theorem 2 shows 
V to be independent of T. = : 


V= 


2. TESTING THE PARAMETERS OF A NORMAL 
DISTRIBUTION 


The four hypotheses o < 99, 0 > 0, 


standard (likelihood-ratio) tests are UMP unbiased for the above four 
Some of the corresponding two-sided problems. 
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For varying £ and ø, the densities 


2 
(8) (2m0?) "enp - 5e - 5 Eat + £x) 


of a sample X,,..., X, from N(£, a?) constitute a two-parameter exponen- 
tial family, which coincides with (1) for 


1 ng 2 A 2357 
dabas 71. dima: U(x) m x2. T(x)- x» FER 


By Theorem 3 of Chapter 4 there exists therefore a UMP unbiased test of 
the hypothesis @ > 6), which for 6 = —1/20¢ is equivalent to H : o > oy. 
The rejection region of this test can be obtained from (12) of Chapter 4, 
with the inequalities reversed because the hypothesis is now 0 > @). In the 
present case this becomes 


Lx} < Q(z) 
where 
P {EX < Co(X)|¥} = a. 
If this is written as 
Dx} - nk? < C (3), 


it follows from the independence of EX? — nX? = X(X, - X)? and X 
(Example 1) that Cj() does not depend on x. The test therefore rejects 
when (x, — x)? < Cj, or equivalently when 


(9) Pus. S C 


9o 


with C determined by P,(X(X, — X)!/o < Co} =a. Since X X, — 
Xy, og has a x Adin bution ‘with n — 1 degrees of freedom, the determin- 
ing condition for C, is 


(10) [i 7 « 


Where x? , denotes the density of a x? variable with n — 1 degrees of 
freedom. 
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The same result can be obtained through Theorem 1. A statistic V — 
h(U, T) of the kind required by the theorem—that is, independent of X for 
9 — o, and all £—is 


y -Y(X,- Xy = U=nT?. 


This is in fact independent of X for all £ and o°. Since h(u, 1) is an 
increasing function of u for each t, it follows that the UMP unbiased test 
has a rejection region of the form V < Cj. 

This derivation also shows that the UMP unbiased rejection region for 
H: ø < 0, or o > o, is n . 


(11) C, sN- xy « c 
where the C’s are given by 

12 O/o 2. = (e. = 
(12) op a0) & = (PL) dy = a. 


Since A(u, t) is linear in u, it is further seen that the UMP unbiased test 
of H : o = oy has the acceptance region 


-12 
xi- 
(13) EA Ue 
% 


with the constants determined by 


1 b re 
(14) [6004 * — [nd (9) dy = =a. 


. This is just the test obtained in Example 2 of Chapter 4 with X(x,- x)? 
in place of Dx? f 


been foreseen, Theorem 1 shows for this and the other hypotheses consid- 


ter exponential family, which were solved previously. 


The power of the above tests can be obtained explicitly in terms of the 


a, a cea 
x -distribution. In the case of the one-sided test (9) for example, it is given 
M 


E(X -X ce yo? 
B(c) -p20 m. 2 = [8 ety) dy. 


5.2] TESTING THE PARAMETERS OF A NORMAL DISTRIBUTION 195 


The same method can be applied to the problems of testing the hypothe- 
ses < £y against > and £— £, against £#&). As is seen by 
transforming to the variables X, — £y, there is no loss of generality in 
assuming that o = 0. It is convenient here to make the identification of (8) 
with (1) through the correspondence 


1 
e- 5, d= = 552" U(x) = x; Bx) = ¥ x2. 


Theorem 3 of Chapter 4 then shows that UMP unbiased tests exist for the 
hypotheses ð < 0 and 0 = 0, which are equivalent to < 0 and £ — 0. 
Since 


X U 
Tea (rues 


is independent of T= XX? when ¿= 0 (Example 1), it follows from 
Theorem 1 that the UMP unbiased rejection region for H:  < 0 is V > Cj 
or equivalently 


(15) #(x) 2. ,, 
where 


ynx 


(16) (x) AA. 
-— L(x, = xy 
n 


In order to apply the theorem to H’: £ = 0, let W = X/ yEx,. This is 
also independent of XX? when £ = 0, and in addition is linear in U = X. 
The distribution of W is symmetric about 0 when £ — 0, and conditions (4), 
(5), (6) with W in place of V are therefore satisfied for the rejection region 
|w| > C' with P,_o{|W]| = C’} = a. Since 


T (n — 1)n W(x) 
t(x) = HEF —() : 


the absolute value of #(x) is an increasing function of |W(x)|, and the 
Tejection region is equivalent to 


(17) |t(x)|2 C. 
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From (16) it is seen that +(X) is the ratio of the two independent random 
variables yn X/o and y/X( X, — X)°/(n — 1)o?. The denominator is dis- 
tributed as the square root of a x?-variable with n — 1 degrees of freedom, 
divided by n — 1; the distribution of the numerator, when £ — 0, is the 
normal distribution N(0,1). The distribution of such a ratio is Student's 


t-distribution with n — 1'degrees of freedom, which has probability density 
T(in) 1 


1 
(18) t.a») = y(n —1) T[i(n- 1)] | y? " 


1+ 


The distribution is symmetric about 0, and the constants C, and C of the 
one- and two-sided tests are determined by 


(19) [0d = and [309 = 5. 


which is then also different from zero. If X/ = cX, and the X, are distrib- 
wied as N(£, a7), the variables X/ have distribution N(£o7). Also 
t( X) = t( X^), and hence t( X") has the same distribution as t( X), as was to 
be proved. [Tables of the power of the t-test are discussed for example, in 
[pos bn Section 7 of Johnson and Kotz (1970, Vol. 2] 

1 denotes any alternative - = 
depends on c. As a 0,8 os rd E 


B(&.o) > f(0) = B(0, 0) = a, 
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The tests of the more general hypotheses £ < £y and £ = £, are reduced 
to those above by transforming to the variables X, — £y. The rejection 
regions for these hypotheses are given as before by (15), (17), and (19), but 
now with 


Yn (x "x £o) 


(x) e AR 
J 1 ES) 
acp 50 x) 


It is seen from the representation of (8) as an exponential family with 
0 — n£/a? that there exists a UMP unbiased test of the hypothesis a < 
£/a? < b, but the method does not apply to the more interesting hypothesis 
a<£<b;* nor is it applicable to the corresponding hypothesis for 
the mean expressed in o-units: a < £/o < b, which will be discussed in 
Chapter 6. 

When testing the mean £ of a normal distribution, one may from 
extensive past experience believe ø to be essentially known. If in fact ø is 
known to be equal to og, it follows from Problem 1 of Chapter 3 that there 
exists a UMP test à; of H:  < £y against K : £ > £o, which rejects when 
(X —£,)/ey is sufficiently large, and this test is then uniformly more 
Powerful than the ż-test (15). On the other hand, if the assumption o = o is 
in error, the size of $o will differ from a and may greatly exceed it. Whether 
to take such a risk depends on one's confidence in the assumption and the 
gain resulting from the use of $o when o is equal to oy. A measure of this 
gain is the deficiency d of the t-test with respect to o, the number of 
additional observations required by the t-test to match the power of $o 
when o = o. Except for very small n, d is essentially independent of 
sample size and for typical values of a is of the order of 1 to 3 additional 
Observations. [For details see Hodges and Lehmann (1970). Other ap- 
proaches to such comparisons are reviewed, for example, in Rothenberg 
(1984).] 


3. COMPARING THE MEANS AND VARIANCES OF TWO 
NORMAL DISTRIBUTIONS 


The problem of comparing the parameters of two normal distributions 
arises in the comparison of two treatments, products, etc., under conditions 
similar to those discussed in Chapter 4 at the beginning of Section 5. We 
consider first the comparison of two variances a? and 7?, which occurs for 
example when one is concerned with the variability of analyses made by two 


"This problem is discussed in Section 3 of Hodges and Lehmann (1954). 
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different laboratories or by two different methods, and specifically the 
hypotheses H: 7/0? < A, and H’: 17/0” = Ao. 
Let X = (X,,..., Xm) and Y = (Y,,..., Y,) be samples from the normal 
distributions N(£, o?) and N(m, 7?) with joint density 
1 1 meé- nm 
Clé, n, o, ex- 34214-32194 et an 


This is an exponential family with the four parameters 


and the sufficient statistics 
DE S a e tasty os da m, a S rest A 


It can be expressed equivalently (see Lemma 2 of Chapter 4), in terms of the 
parameters 


=- + —, 9*-9, (i21,2,3 
z d T ie:1:2:3) 
and the statistics 
1 = 
Um O Tia wit rye aoe teg s13 =X. 


The hypotheses 6* <0 and 6* — 0, which are equivalent to H and 


H' respectively, therefore possess UMP unbiased tests by Theorem 3 of 
Chapter 4. 


When 7? = A,o?, the distribution of the statistic 


does not depend on o & or i i 
: » $ Or m, and it follows from Corollary 1 that V is 
independent of (T, T*, T;*). The UMP unbiased test of H is therefore 
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given by (2) and (3), so that the rejection region can be written as 


E(X- Y)/A,(n- 1) R 
LGE mS ~ 


When 7? = A,o?, the statistic on the left-hand side of (20) is the ratio of the 
two independent x? variables L(Y, — Y)?/r? and D(X, — X)?/o?, each 
divided by the number of its degrees of freedom. The distribution of such a 
ratio is the F-distribution with n — 1 and m — 1 degrees of freedom, which 
has the density 


(20) 


T[i(m4n- 2)] 


n—] yen 
€). Fama) = Fe DITO i) 


yrr-D-1 


x nem im*n-2)' 
(: + ») 


m=i 
The constant C of (20) is then determined by 


Q2) [Fermi dy — a. 


In order to apply Theorem 1 to H' let 


NIS om 
s E(x,- X + Q/A)X Y). 


This is also independent of T* = (T;*, T^, T;*) when 7? = Ao”, and is 
linear in U*. The UMP unbiased acceptance region of H' is therefore 


(23) CGsWzC, 


with the constants determined by (5) and (6) where V is replaced by W. On 
dividing numerator and denominator of W by o? it is seen that for 
T? = Aoc? the statistic W is a ratio of the form W,/(W, + W;), where W, 
and W, are independent x? variables with n — 1 and m — 1 degrees of 
freedom respectively. Equivalently, W = Y/(1 + Y), where Y= W/W, 
and where (m — 1)Y/(n — 1) has the distribution F, ; ,,..,. The distribu- 
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tion of W is the beta-distribution* with density 
(24) 


T[i(m +n- 2)] 


SLANE Saas Cn 3)/4 y) 
Seat ph omn]. oc 


0<w<l. 


The conditions (5) and (6), by means of the relations 


ni 
Ue mtn-2 
and 
DE 
9Byos n san) = =y p Bion dona?) 
become 


C; 'C; 
(25) [Buen aes ow) dv T [Biossae 0?) dw — 1— a. 
1 1 


a Te definition of V shows that its distribution depends only on the ratio 
T/a", and so does the distribution of W. The power of the tests (20) and 
(23) is therefore also a function only of the variable ^ — 1?/0?; it can be 


ri aig explicitly in terms of the F-distribution, for example in the first 
case by 


X(Yy-Yy^»-1 GA 
BA) m pu CE ved) 
o» E(QX,-Xf/é(m-1)^ A 


T Fetal) dy. 


ee AYER of egudtity of the means £, ņ of two normal distributions 
with unknown variances o? and 7?, the so-called Behrens—Fisher problem, is 


*The relationship W = Y/(1 + Y) shows the F- an istributi i 
xr F - and beta-distributions to be equivalent. 
Tables of these distributions are discussed in Chapters 24 and 26 of Johnson and Kotz (1970, 


Vol. 2). Critical values of F are tabled by Mardia i 
algorithms for the associated Kelsi teins noua re 
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not accessible by the present method. (See Example 5 of Chapter 4; for a 
discussion of this problem see the next section and Chapter 6, Section 6.) 
We shall therefore consider only the simpler case in which the two variances 
are assumed to be equal. The joint density of the X's and Y’s is then 


1 £ 
(26) C(&, n, a)exp BO aie Y») + gibt E» $ 


which is an exponential family with parameters 


£ 1 


" 
pores aane ed 


and the sufficient statistics 
U-LY, Mpa mae + Ye. 
For testing the hypotheses 
H:q-£sx0 and H’:n-£§=0 


it is more convenient to represent the densities as an exponential family 
with the parameters 


2-6 Lo fid ota yang 
m n 


and the sufficient statistics 


U*-Y-X, Tt-mX4nY, T= LX? + XY. 


That this is possible is seen from the identity 


(y-X)»-£) | (mx + ny)(mé + nn) 
méX + nny = T T $F Qux UWEDP NU 
mn 


It follows from Theorem 3 of Chapter 4 that UMP unbiased tests exist for 
the hypotheses 6* < 0 and 6* = 0, and hence for H and H’. 
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When 7 = £, the distribution of 


Y-X 
y- E =F 
EE XY + EG, - Y) 
UF 
y 1 mn 
T* = T* — Ue 
S mn! mtn 


does not depend on the common mean £ or on c, as is seen by replacing X; 
with (X, — £)/c and Y, with (Y, — £)/c in the expression for V, and V is 
independent of (7;*, T;*). The rejection region of the UMP unbiased test of 
H can therefore be written as V 2 C; or 


(27) (X, Y) 2 Cy, 


where 


Q8) (X.Y) BENE 
28) (x, Y) = —————————————————. 
Y L(x, -X+ Xr - ¥)|/(m + n - 2) 


The statistic (( X, Y) is the ratio of the two independent variables 


qs - 


The numerator is normally distributed with mean (y — £)/ ym! + nc 
and unit variance; the denominator, as the square root of a x? variable with 
m + n — 2 degrees of freedom, divided by m + n — 2. Hence t(X, Y) has a 


noncentral t-distribution with m + n — 2 degrees of freedom and non- 
centrality parameter 
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When in particular ņ — £ — 0, the distribution of t( X, Y) is Student's 
t-distribution, and the constant C, is determined by 


(29) [tese = a. 


As before, the. assumptions required by Theorem 1 for H' are not 
satisfied by V itself but by a function of V, 


Y-X 


W= n 


m+n 


which is related to V through 


Since W is a function of V, it is also independent of (T;*, T;*) when y = £; 
in addition it is a linear function of U* with coefficients dependent only on 
T*. The distribution of W being symmetric about 0 when 7 = £, it follows, 
as in the derivation of the corresponding rejection region (17) for the 
one-sample problem, that the UMP unbiased test of H’ rejects when |W| is 
too large, or equivalently when 


(30) |t(X,Y)|> C. 


The constant C is determined by 
do a 
[ita sual) dy mate 
Cc 


The power of the tests (27) and (30) depends only on (n — £)/c and is 
given in terms of the noncentral 1-distribution. Its properties are analogous 
to those of the one-sample t-test (Problems 1, 2, and 4). 


4. ROBUSTNESS 


Optimality theory postulates a statistical model and then attempts to 
determine a best procedure for that model. Since model assumptions tend to 
be unreliable, it is necessary to go a step further and ask how sensitive the 
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procedure and its optimality are to the assumptions. In the normal models 
of the preceding section, three assumptions are made: Independence, iden- 
tity of distribution, and normality. In the two-sample /-test, there is the 
additional assumption of equality of variance. We shall consider the effects 
of nonnormality and inequality of variance in the present section, and that 
of dependence in the next. 

The natural first question to ask about the robustness of a test concerns 
the behavior of the significance level. If an assumption is violated, is the 
significance level still approximately valid? Such questions are typically 
answered by combining two methods of attack: The actual significance level 
under some alternative distributions is either calculated exactly or, more 
usually, estimated by simulation. In addition, asymptotic results are ob- 
tained which provide approximations to the true significance level for a wide 
variety of models. 

We here restrict ourselves to a brief sketch of the latter approach. For 
this purpose we require the following basic results from probability theory. 
[For a more detailed discussion, see for example Cramér (1946); TPE, 
Chapter 5; and Serfling (1980).] The first is the simplest form of the central 
limit theorem. 


Theorem 3. (Central limit theorem.) Let X,,..., X, be independently 
identically distributed with mean E(X,) = &£ and Var(X;) = o? < oo. Then 


for all real t 
až- 
p209 ES i cm) $(1), 


where ® denotes the cumulative distribution function of the standard normal 
distribution N(0, 1). 


When the cumulative distribution functions of a sequence of random 
variables T, tend to a continuous limiting cumulative distribution function 
G as above, we shall say that T, converges to G in law. If T, and T/ are 
independent and converge to N(a, b?) and N(a', b?) respectively, then 
7: s D. converges to N(a + a’, b? + 5’), 

n converges in law to N(0,1), then bT, + a (b + 0) converges in law 
to N(a, b). The following result concerns the Rai be limit. behavior 


when a and b are replaced by rando i i i 
Mies y m variables which tend to a and b in 


Theorem 4. Af T, converges in law to some distribution G and if A,, B, 
are random variables converging in probability to a and b + 0 respectively, 
then B,T, + A, has the same limit distribution as bT, + a. 
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Corollary 2. If T, tends in law to G (continuous) and if c, > G, then 


P(T, x c,) > G(c). 


The last of the auxiliary results concerns the asymptotic behavior of 
functions of asymptotically normal variables. 


Theorem 5. If T, is a sequence of random variables for which yn. Inm —9) 
tends in law to N(0, 7°), then for any function f for which f'(8) exists and is 
* 0, 


Yn [ f(T,) - f(8)] 


tends in law to N(0, 77[ f'(8)]2). 


Consider now the one-sample problem of Section 2, so that X;,..., X, 
are independently distributed as N(£, o°). Tests of H : £ — £, are based on 
the test statistic 


(Ais alee 5 Ea 


where $? = Y( X, — X)?/(n — 1). When £ = , and the X’s are normal, 
1(X) has the t-distribution with n — 1 degrees of freedom. Suppose, how- 
ever, that the normality assumption fails and the X’’s instead are distributed 
according to some other distribution F with mean £, and finite variance. 
Then by Theorem 3, /n(X —£,)/e has the limit distribution N(0, 1); 
furthermore S/o tends to 1 in probability (see, for example, TPE, Chapter 
5). By Theorem 4, t(X) therefore has the limit distribution N(0, 1) regard- 
less of F. This shows in particular that the ¢-distribution tends to N(0, 1) as 
n> oco. 

To be specific, consider the one-sided t-test which rejects when t( X) > Cn 
Where P(t( X) > C,) = a when F is normal. It follows from Corollary 2 
and the asymptotic normality of the ¢-distribution that 


C >u, = 0 (1-a). 


(If this were not the case, a subsequence of the C, would converge to a 
different limit, and this would lead to a contradiction.) 

Let a,,(F) be the true probability of the rejection region t > C, when the 
distribution of the X’s is F. Then a,(F) = Pr{t 2 C,} has the same limit 
as P,{t > u,}, which is a. For sufficiently large n, the actual size a,(F) 
will therefore be close to the nominal level a; how close depends on F and 
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n. For entries to the literature dealing with this dependence, see Cressie 
(1980), Tan (1982), and Benjamini (1983). 

To study the corresponding test of variance, suppose first that the mean £ 
is 0. When F is normal, the UMP test of H : o = a, against ø > o, rejects 
when X X2/ad is too large, where the null distribution of XX2/oj is x7. By 
Theorem 3, Vn (£X? — noZ)/n tends in law to N(0,20,)) as n > co, since 
Var( X7) = 20j. If the rejection region is written as 


EX? — nog * 
ror 


it follows that C, > ty 

Suppose now instead that the X's are distributed according to a distri- 
bution F with E(X,) = 0, E(X?) = Var X, = o°, and Var X? = y?. Then 
E(X? — noj)/ Vn tends in law to N(0, y?) when o = o, and the size 
a, ( F) of the test tends to 


YX?- no? u,V2 og 
Impe ues 
| Vine > uy 1 o| = | 


Depending on y, which can take on any positive value, the sequence a,(F) 
can thus tend to any limit < 3. Even asymptotically and under rather small 
departures from normality (if they lead to big changes in y), the size of the 
x^-test is thus completely uncontrolled. 

For sufficiently large n, the difficulty is easy to overcome. Let Y; = X7, 
E(Y,) = = o°, The test statistic then reduces to yn (Y —n9). To obtain an 
asymptoticaly valid test, it is only necessary to divide by a suitable 
estimator of /Var Y, such as yX(Y, — Y)’/n. (However, since Y? = Xf, 
small changes in the tail of X, may have large effects on Y?, and n may 
ral be rather large for the asymptotic result to give a good approxima- 


_ When £ is unknown, the normal theory test for o? i E: 
Y iM ERE ry test for o? is based on X X; 


1 yv 
a EG - X) - nog] - EX - no?) - "S 


2). To see this, note that the distri- 


again has the limit distribution N(0, y 
independent of £, and put £ — 0. Since /n X hasa 


bution of E(X, — X)? is 
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(normal) limit distribution, nX? is bounded in probability,* and nX?/ Vn 
tends to zero in probability. The result now follows from that for £ = 0 and 
Theorem 4. 

The above results carry over to the corresponding two-sample problems. 
For the t-test, an extension of the one-sample argument shows that as 
m, n > œ, (Y — X)/ /1/m + 1/no tends in law to N(0, 1) while [Z(X, — 
X)? + L(Y, — Y)?]/(m + n — 2)o? tends in probability to 1 for samples 
X,,..., Xm Yp ---, Y, from any common distribution F with finite vari- 
ance. Thus, the actual size a,, ,(F) tends to a for any such F. 

On the other hand, the F-test for variances, just like the one-sample 
x?-test, is extremely sensitive to the assumption of normality. To see this, 
express the rejection region in terms of log SẸ — log $7, where Sz = X X, 
— X)?/(m — 1) and S; = X(Y, — Y)?/(n — 1), and suppose that as m 
and n > oo, m/(m + n) remains fixed at p. By the result for the one-sam- 
ple problem and Theorem 5 with f(u) = log u, it is seen that ym [log SẸ — 
log c?] and yn [log S2 — log o°] both tend in law to N(0, y?/o^) when the 
X's and Y’s are distributed as F, and hence that ym + n [log SẸ — log SẸ] 
tends in law to the normal distribution with mean 0 and variance 


y[1 Al y? 
P eae | EVE 
o*\p 1-p} p(l-p)o 


In the particular case that F is normal, y? = 20% and the variance of the 
limit distribution is 2/p(1 — p). For other distributions y?/o* can take on 
any positive value and, as in the one-sample case, a,(F) can tend to any 
limit < 1. [For an entry into the extensive literature on more robust 
alternatives, see for example Conover, Johnson, and Johnson (1981) and 
Tiku and Balakrishnan (1984).] - 

Having found that the size of the one- and two-sample t-tests is relatively 
insensitive to nonnormality (at least for large samples), let us turn to the 
Corresponding question concerning the power of these tests. By similar 
asymptotic calculations, it can be shown that the same conclusion holds: 
Power values of the t-tests obtained under normality are asymptotically 
valid also for all other distributions with finite variance. This is a useful 
result if it has been decided to employ a t-test and one wishes to know what 
Power it will have against a given alternative £/o or (y — £)/o, or what 
Sample sizes are required to obtain a given power. — 

It is interesting to note that there exists a modification of the test, 
Whose size is independent of F not only asymptotically but exactly, and 


“See, for example, TPE, Chapter 5, Problem 1.24. 
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whose asymptotic power is equal to that of the t-test. This permutation 
version of the t-test will be discussed in Sections 10-14. It may seem that 
such a test has all the properties one could hope for. However, this 
overlooks the basic question of whether the t-test itself, which is optimal 
under normality, will retain a high standing with respect to its competitors 
under other distributions. The /-tests are in fact not robust in this sense. 
Tests which are preferable when a broad spectrum of distributions F is 
considered possible will be discussed in Chapter 6, Section 9. A permutation 
test with this property has been proposed by Lambert (1985). 

The above distinction between robustness of the performance of a given 
test and robustness of its relative efficiency with respect to alternative tests 
has been pointed out by Tukey and McLaughlin (1963) and Box and Tiao 
(1964), who have described these concepts as robustness of validity or 
criterion robustness, and as robustness of efficiency or inference robustness, 
respectively. 

As a last problem, consider the level of the two-sample t-test when 
the variances Var( X,) = o? and Var(Y)) = 7? are in fact not equal. As be- 
fore, one finds that (Y —X)/o7/m + 1?/n tends in law to N(0,1) as 
m, n co, while Sz = E(X; — X)?/(m — 1) and $? = X(Y, - Y)?/(n - 
1) respectively tend to o? and 7? in probability. If m and n tend to oo 


through a sequence with fixed proportion m/(m +n) — p, the squared 
denominator of t, 


m-1 n-1 
D re Reb bp I ed eB) 
Web SO el a ERA P^ 


tends in probability to po? + (1 — p)7?, and the limit of 


o? T 
Saivi UT —+— 
ant T Y-X 1 m n 
1 2 2 
Saas Buea acetates ac 
m m m n 
is normal with mean zero and variance 
(1) (1 - p)o? + pr? 


po? + (L= p)ct 


When m = n, so that p = 


4, the tt : 2 
level even if ø and 7 are far $ est thus has approximately the right 


apart. The accuracy of this approximation for 
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different values of m=n and 1/o is discussed by Ramsey (1980) and 
Posten, Yeh, and Owen (1982). However, when p * 3, the actual size of the 
test can differ greatly from the nominal level a even for large m and n. An 
approximate test of the hypothesis H: ņn = £ when o, 1 are not assumed 
equal (the Behrens-Fisher problem), which asymptotically is free of this 
difficulty, can be obtained through Studentization*, i.e., by replacing D? 
with (1/m)S2 + (1/n)S2 and referring the resulting statistic to the stan- 
dard normal distribution. This approximation is very crude, and not reliable 
unless m and n are fairly large. A refinement, the Welch approximate t-test, 
refers the resulting statistic not to the standard normal but to the :-distribu- 
tion with a random number of degrees of freedom f given by 


1 RUE al 1 

- = | —— | +t OO 

f t (GEER) nea 
where 


_ Q/m)si , 
— (1/n)S}° 


When the X’s and Y's are normal, the actual level of this test has been 
shown to be quite close to the nominal level for sample sizes as small as 
m= 4, n = 8 and m =n = 6 [see Wang (1971)]. A further refinement will 
be mentioned in Chapter 6, Section 6. Ne 

The robustness of the level of Welch’s test against nonnormality is 
studied by Yuen (1974), who shows that for heavy-tailed distributions the 
actual level tends to be considerably smaller than the nominal level (which 
leads to an undesirable loss of power), and who proposes an alternative. 
Some additional results are discussed in Scheffé (1970) and in Tiku and 
Singh (1981). The robustness of some quite different competitors of the 
t-test is investigated in Pratt (1964). 


5, EFFECT OF DEPENDENCE 


The one-sample t-test arises when a sequence of measurements X, ..., X, is 
taken of a quantity £ and the X’s are assumed to be independently 


distributed as N(£, o?). The effect of nonnormality on the level of the test 
was discussed in the preceding section. Independence may seem like a more 
innocuous assumption. However, it has been found that observations occur- 


*Studentization is defined in a more general context at the end of Chapter 7, Section 3. 
* For a variant, see Fenstad (1983). 
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ring close in time or space are often positively correlated [Student (1927), 


Hotelling (1961), Cochran (1968). The present section will therefore be 
concerned with the effect of this type of dependence. 


Lemma 1. Let X,,..., X, be jointly normally distributed with common 
marginal distribution N(0, a?) and with correlation coefficients p;, = 
corr( X;, X;). As n > co, suppose that 


A o n n 
(a) VarX=— 5, 9,0, > 0, 
n` i=l j=l 
1 
(b) Var( £x) 20 
and 
1 
(c) 22, 0j r^g. 
n inj 
Then 


(i) the distribution of the t-statistic (16) tends to the normal distribution 
N(0,1 + y); y 
Cii) if y #0, the level of the t-test is not robust even asymptotically as 


n > 0. Specifically, if y > 0, the asymptotic level of the t-test carried out at 
nominal level « is 


1-e| 


Proof. (i): Since the X, are jointly normal, the numerator yn X of t is 
also normal, with mean zero and variance 


Var(Vn X) = eh + EE) 


and hence tends in law to N(0,o 


square root of "(1 + y). The denominator of 1 is the 


D? = 


n 
2s your dro 
ueni Aus 
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It follows from the Chebyshev inequality (Problem 18) that X: X2/(n — 1) 
tends in probability to E(X?) = e? and [n/(n — 1)]X? to zero, so that 
D > o in probability. By Theorem 4, the distribution of t therefore tends to 
N(0,1 4 y). 

The implications (ii) are obvious. 

Under the assumptions of Lemma 1, the joint distribution of the X's is 
determined by o? and the correlation coefficients p,,, with the asymptotic 
level of the t-test depending only on y. The following examples illustrating 
different correlation structures show that even under rather weak depen- 
dence of the observations, the assumptions of Lemma 1 are satisfied with 
y * 0, and hence that the level of the t-test is quite sensitive to the 
assumption of independence. 


MoprL A. (CLUSTER SAMPLING). Suppose the observations occur in 5 
groups (or clusters) of size m, and that any two observations within a group 
have a common correlation coefficient p, while those in different groups are 
independent. (This may be the case, for instance, when the observations 
within a group are those taken on the same day or by the same observer, or 
involve some other common factor.) Then (Problem 20) 


2 


Var X = — [1 + (m - 1)e]. 


which tends to zero as s — oo; and analogously assumption (b) is seen to 
hold. Since y = (m — 1)p, the level of the t-test is not asymptotically robust 
as s > oo. In particular, the test overstates the significance of the results 
when p > 0. 

To provide a specific structure leading to this model, denote the observa- 
tions in the ith group by X;; (j=1,.:.,m), and suppose that X= A, + 
U,,, where A, is a factor common to the observations in the ith group. If the 
A’s and U’s (none of which are observable) are all independent with normal 
distributions N(£, o2) and N(0, oj) respectively, then the joint distribution 
of the X’s is that prescribed by Model A with o? = o} + aj and p = 04/0". 


MopzL B. (MOVING-AVERAGE PROCESS). When the dependence of 
nearby observations is not due to grouping as in Model A, it is often 
reasonable to assume that p;, depends only on |j — i| and is nonincreasing 
in |j — i|. Let p; ;,, then be denoted by p,, and suppose that the correla- 
tion between X, and X,, , is negligible for k > m (m an integer < n), so 
that one can put p, = 0 for k > m. Then the conditions for Lemma 1 are 
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satisfied (Problem 22) with 
m 
Y-2 Y p. 
k=l 


In particular, if p,,..., p, are all positive, the t-test is again too liberal. 
A specific structure leading to Model B is given by the moving-average 
process 


X= E+ E BUj 
j=0 


where the U’s are independent N(0, og). The variance o? of the X's is then 
= op Eob? and 


m-k 
Y BB... 
p, dem fo k< nm, 
s 
0 for k»m. 


Mopzr C. (FIRST-ORDER AUTOREGRESSIVE PROCESS). A simple model 
for dependence in which the le,| are decreasing in k but + 0 for all k is 
the first-order autoregressive process defined by 


Xii = E+ B(X,- E) + Uys, IBI Py. n 


with the U, independent N(0, o2). If X, is N(£, 7°), the marginal distri- 
bution of X, for i > 1 is normal with mean £ and variance o? = B?o? , + oj. 
The variance of X, will thus be independent of i provided 7? — o¢/(1 — B?). 
For the sake of simplicity, we shall assume this to be the case, and take £ to 
be zero. From 


Xa BEX, + Bes ii BETAU iS DET * BU, La + Usk 

: t ; B", so that the correlation between X, and X, 

Me ee with increasing | j — i]. The assumptions of Lemma 
are again satisfied, and y = 2B/(1 — B). Thus, in this case too, the level 


asymptotic level when the nominal level i 
and Rubin (1971).] 
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It is seen that in general the effect of dependence on the level of the t-test 
is more serious than that of nonnormality. Unfortunately, it is not possible 
to robustify the test against general dependence through Studentization, as 
. ean be done for unequal variances in the two-sample case. This would 
require consistent estimation of y and hence of the Pij, Which is unavailable, 
since the number of unknown parameters far exceeds the number of 
Observations. 

The difficulty can be overcome if enough information is available to 
reduce the general model to one, such as A-C,* depending only on a finite 
number of parameters which can then be estimated consistently. Some 
specific procedures of this type are discussed by Albers (1978), [and for an 
associated sign test by Falk and Kohne (1984)]. Such robust procedures will 
in fact often also be insensitive to the assumption of normality, as can be 
shown by appealing to an appropriate central limit theorem for dependent 
variables [see e.g. Billingsley (1979)]. The validity of these procedures is of 
course limited to the particular model assumed, including the value of a 
parameter such as m in Models A and B. 

The results of the present section easily extend to the case of the 
two-sample t-test, when each of the two series of observations shows 
dependence of the kind considered here. 


6. CONFIDENCE INTERVALS AND FAMILIES OF TESTS 


Confidence bounds for a parameter @ corresponding to a confidence level 
1 — a were defined in Chapter 3, Section 5, for the case that the distribution 
of the random variable X depends only on 0. When nuisance parameters ® 
are present the defining condition for a lower confidence bound 0 becomes 


(32) P,Q(8(X)s0)21-a forall 0,9. 


Similarly, confidence intervals for at confidence level 1 — a are defined as 
a set of random intervals with end points &( X), 0( X) such that 


(33) Py {8(X) 6x 8(X)) 1-2 forall 0, ð. 


The infinum over (8, 9) of the left-hand side of (32) and (33) is the 

confidence coefficient associated with these statements. : 
As was already indicated in Chapter 3, confidence statements permit a 

dual interpretation. Directly, they provide bounds for the unknown parame- 


*Models of a sequence of dependent observations with various covariance Structures are 
discussed in books on time series such as Anderson (1971) and Box and Jenkins (1970). 


214 UNBIASEDNESS: APPLICATIONS; CONFIDENCE INTERVALS [5.6 


ter 0 and thereby a solution to the problem of estimating 0. The statement 
x 0 < 8 is not as precise as a point estimate, but it has the advantage that 
the probability of it being correct can be guaranteed to be at least 1 — a. 
Similarly, a lower confidence bound can be thought of as an estimate 6 
which overestimates the true parameter value with probability <a. In 
particular for a = 2, if @ satisfies 


Py o(9< 0} = Py o(8 20} =}, 


the estimate is as likely to underestimate as to overestimate and is then said 
to be median unbiased. (See Chapter 1, Problem 3, for the relation of this 
property to a more general concept of unbiasedness.) For an exponential 
family given by (10) of Chapter 4 there exists an estimator of 0 which 
among all median unbiased estimators uniformly minimizes the risk for any 
loss function L(@, d) that is monotone in the sense of the last paragraph of 
Chapter 3, Section 5. A full treatment of this result including some prob- 
abilistic and measure-theoretic complications, is given by Pfanzagl (1979). 
Alternatively, as was shown in Chapter 3, confidence statements can be 
viewed as equivalent to a family of tests. The following is essentially a 
review of the discussion of this relationship in Chapter 3, made slightly 
more specific by restricting attention to the two-sided case. For each 6, let 
4(0,) denote the acceptance region of a level-a test (assumed for the 
moment to be nonrandomized) of the hypothesis H(6,) : 0 = 6,. If 


S(x) = {0:x e 4(0)) 


then 

(34) 6€S(x) ifandonlyif x € A(6), 
and hence 

(35) Py {9ES(X)}>1-a forall 0, 9. 


Thus any family of level-a acceptance regions, through the correspondence 
(34), leads to a family of confidence sets at confidence level 1 — a. 


Conversely, given any class of confidence sets S(x) satisfying (35), let 
(36) A(6) = {x:0 € s(x)). 


Den E o A(9) are level-a acceptance regions for testing the hypotheses 
(85) : 8 = 6, and the confidence sets S(x) show for each 6) whether for 


Hibs obra x observed the hypothesis 6 = 6, is accepted or rejected at 
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Exactly the same arguments apply if the sets A(6)) are acceptance 
regions for the hypotheses @ < . As will be seen below, one- and two-sided 
tests typically, although not always, lead to one-sided confidence bounds 
and to confidence intervals respectively. 


Example 4. Normal mean. Confidence intervals for the mean £ of a normal 
distribution with unknown variance can be obtained from the acceptance regions 
A(£5) of the hypothesis H : £ = £y. These are given by 


|n (x — &)| 


Wine EN 
y X Gi - x) (n - 1) 


Where C is determined from the t-distribution so that the probability of this 
inequality is 1 — a when £ = £y. [See (17) and (19) of Section 2.] The set S(x) is 
then the set of £'s satisfying this inequality with £ = £,, that is, the interval 


(37) id EG-» s EG». 


The class of these intervals therefore constitutes confidence intervals for £ with 
confidence coefficient 1 — a. 


mv i 

The length of the intervals (37) is proportional to yX( x; » x) s and their 
expected length to c. For large o, the intervals will therefore provide little informa- 
tion concerning the unknown £. This is a consequence of the fact, which led to 
similar difficulties for the corresponding testing problem, that two normal distribu- 
tions N(£,, o?) and N(£,,o?) with fixed difference of means become indis- 
tinguishable as ø tends to infinity. In order to obtain confidence intervals for £ 
Whose length does not tend to infinity with c, it is necessary to determine the 
number of observations sequentially so that it can be adjusted to c. A sequential 
pevepitting leading to confidence intervals of prescribed length is given in Problems 

6 and 27. d 

However, even such a sequential procedure does not really dispose of the 
difficulty, but only shifts the lack of control from the length of the interval to the 
number of observations. As g — oo, the number of observations required to obtain 
Confidence intervals of bounded length also tends to infinity. Actually, in LS 
One will frequently have an idea of the order of magnitude of o. With a d le 
cither of fixed size or obtained sequentially, it is then necessary to establish a 
balance between the desired confidence 1 — a, the accuracy given by the length / of 
the interval, and the number of observations n one is willing to expend. In such an 
arrangement two of the three quantities 1 — a, /, and n will be fixed, while the hs 
is a random variable whose distribution depends on c, so that it will be less wel 
Controlled than the others. If 1 — a is taken as fixed, the choice between a 
Sequential scheme and one of fixed sample size thus depends essentially on whether 
it is more important to control / or n. 
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To obtain lower confidence limits for £, consider the acceptance regions 


yn(X - £o) < 
VE- 3)/(n - 1) 


for testing £ < £, against £ > £y. The sets S(x) are then the one-sided intervals 


i- Sy -—EG-» st 


the left-hand sides of which therefore constitute the desired lower bounds £. If « = }, 
the constant C, is 0; the resulting confidence bound £ — X is a median unbiased 
estimate of £, and among all such estimates it uniformly maximizes 


P(-Mxst£-ts^, forall A,,A,>0. 


(For a proof see Chapter 3, Section 5.) 


7. UNBIASED CONFIDENCE SETS 


Confidence sets can be viewed as a family of tests of the hypotheses 
0 € H(6’) against alternatives 0 € K(6") for varying 6’. A confidence level 
of 1 — a then simply expresses the fact that all the tests are to be at level a, 
and the condition therefore becomes 


(38) P,,(€e€s(X) 21-a forall 0 € H(6') and all 9. 


In the case that H(6’) is the hypothesis 0 = 6’ and S(X) is the interval 
[9 (X), 6( X)], this agrees with (33). In the one-sided case in which H(0^) is 
the hypothesis à < 8’ and S(X) = (0: 8(X) x 0), the condition reduces 
to Pj ,(0(X) < 0') = 1 — a for all 0 > 6, and this is seen to be equivalent 
to (32). With this interpretation of confidence sets, the probabilities 


(39) Pj (0 € S(X)), 0 € K(0^), 


are the probabilities of false acceptance of H(0^) (error of the second kind). 
The smaller these probabilities are, the more desirable are the tests. 

From. the point of view of estimation, on the other hand, (39) is the 
probability of covering the wrong value 6". With a controlled probability of 
covering the true value, the confidence sets will be more informative the less 
likely they are to cover false values of the parameter. In this sense the 


probabilities (39) provide a measure of the accuracy of the confidence sets. 


A justification of (39) in terms of loss functions as gi -sided 
case in Chapter 3, Section 5. C creasti 
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In the presence of nuisance parameters, UMP tests usually do not exist, 
and this implies the nonexistence of confidence sets that are uniformly most 
accurate in the sense of minimizing (39) for all 8’ such that 0 € K(0’) and 
for all 9. This suggests restricting attention to confidence sets which in a 
suitable sense are unbiased. In analogy with the corresponding definition for 
tests, a family of confidence sets at confidence level 1 — a is said to be 
unbiased if 


(40) Po, (0 € S(X)} s1-a 
for all 6’ such that 0 € K(0') and forall 9 and 6, 


so that the probability of covering these false values does not exceed the 
confidence level. 

In the two- and one-sided cases mentioned above, the condition (40) 
reduces to 


Py {8<0' <8} <1-a  forall6' * 0 andall 9 


and 


P,Q,(0x0) <1—a  forall0' « 0 andall 9. 


With this definition of unbiasedness, unbiased families of tests lead to 
unbiased confidence sets and conversely. A family of confidence sets is 
uniformly most accurate unbiased at confidence level 1 — a if it minimizes 
the probabilities 


P, 9{ 8’ € S( X)) for all 6’ such that @ € K(6’) and for all # and 6, 


Subject to (38) and (40). The confidence sets obtained on the basis of the 
UMP unbiased tests of the present and preceding chapter are therefore 
uniformly most accurate unbiased. This applies in particular to the con- 
fidence intervals obtained in the preceding sections. Some further examples 
are the following. 


Xmpi 7 i 2), the 
E le 5. Normal variance. If X,,..., X, is a sample from N(£, 0°), 
UMP unbiased test of the hypothesis a E ay is given by the acceptance region (13) 


v2 
cfg ei oes, 
0 


Where Cf and Cj are determined by (14). The most accurate unbiased confidence 
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intervals for o? are therefore 
1 EEUEM! E 
gala) so SG bls — xy. 


[Tables of Cj and C; are provided by Tate and Klett (1959).] Similarly, from (9) 
and (10) the most accurate unbiased upper confidence limits for a^ are 


1 -y2 
o? sa =x)’, 
where 


[5960 dy 21- a. 


The corresponding lower confidence limits are uniformly most accurate (without the 
restriction of unbiasedness) by Chapter 3, Section 9. 


Example 6. Difference of means. Confidence intervals for the difference A = 
m — & of the means of two normal distributions with common variance are obtained 
from tests of the hypothesis y — £ — A,. If X,,..., X, and Y,,..., Y, are distrib- 
uted as N(£, o?) and N(v, o?) respectively, and if Y/ = Y,- Ao, y = 9 — Ap, the 
hypothesis can be expressed in terms of the variables X, and Y/ as y’ —£ = 0. 
From (28) and (30) the UMP unbiased acceptance region is then seen to be 


TI 
eee 2l ub 
IG o)l Ep 


[xo -9* «EG; -»»]/m «2-2 i 


where C is determined by the equation following (30). The most accurate unbiased 
confidence intervals for ņ — £ are therefore 


C, 


(41) (y-3)-C$s5»-£«(y-x)« cs 


where 


s- ($i) EG ro-» 
m n mtn-2 F 


The one-sided intervals are obtained analogously. 


Example 7. Ratio of variances, If X,,..., X, and Y, Y, are samples from 
N(&, 0?) and N(n, 12), most accurate unbiased Confidence intervals for A = 1/0 


5.7] UNBIASED CONFIDENCE SETS 219 
are derived from the acceptance region (23) as 


1-6 XG;-»* ee ae X6 -»* 
G. Lax GU YE 


(42) 


where C, and C, are determined from (25).* In the particular case that m = n, the 
intervals take on the simpler form 


Sip JY (yos Dy 


LeXOE AA 
kY(u-3y v^ Yy(-zy 


(43) 


where k is determined from the F-distribution. Most accurate unbiased lower 
confidence limits for the variance ratio are 


SIR T 
1 ào-»/0-9 7 


(44) A- 2 


«o GY(-Xx)/m-1) v 


With C, given by (22). If in (22) a is taken to be 1, this lower confidence limit A 
becomes a median unbiased estimate of 7/o?. Among all such estimates it 
uniformly minimizes 


2 
(a, si-asa) forall A,,A,>0. 
o 


(For a proof see Chapter 3, Section 5). 


So far it has been assumed that the tests from which the confidence sets 
are obtained are nonrandomized. The modifications that are necessary when 
this assumption is not satisfied were discussed in Chapter 3. The rando- 
mized tests can then be interpreted as being nonrandomized in the space of 
X and an auxiliary variable V which is uniformly distributed on the unit 
interval. If in particular X is integer-valued as in the binomial or Poisson 
Case, the tests can be represented in terms of the continuous variable 
X + V. In this way, most accurate unbiased confidence intervals can be 
Obtained, for example, for a binomial probability p from the UMP unbiased 
tests of H: p = Po (Example 1 of Chapter 4). It is not clear a priori that the 
Tesulting confidence sets for p will necessarily by intervals. This is, however, 
a consequence of the following Lemma. 


*A comparison of these limits with those obtained from the equal-tails test is given by 
Scheffé (1942); some values of C, and C, are provided by Ramachandran (1958). 
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Lemma 2. Let X be a real-valued random variable with probability density 
P(x) which has monotone likelihood ratio in x. Suppose that UMP unbiased 
tests of the hypotheses H(@): 0 = 0, exist and are given by the acceptance 
regions 


C(b) < x < C, (6), 


and that they are strictly unbiased. Then the functions C0) are strictly 
increasing in 0, and the most accurate unbiased confidence intervals for 0 are 


€3!(x) S8 x Gn(x): 


Proof. Let 6) < 6,, and let B,(0) and B,(0) denote the power functions 
of the above tests ġo and 6, for testing à = 6, and @ = 6. It follows from 
the strict unbiasedness of the tests that 


Eg, [$1 (X) = $0(X)] = &(&) - a > 0 > a — Bo(6,) 


= Ey,[¢,(X) - $9(X)]. 


Thus neither of the two intervals [C,(6,),C,(8,)] (i = 0,1) contains the 
other, and it is seen from Lemma 2(iii) of Chapter 3 that C,(65) < C,(4,) 
for i= 1,2. The functions C, therefore have inverses, and the inequalities 
defining the acceptance region for H(@) are equivalent to C; (x) < 0 < 
C; (x), as was to be proved. 

The situation is indicated in Figure 1. From the boundaries x = C,(0) 
and x = C,(@) of the acceptance regions A(@) one obtains for each fixed 


value of x the confidence set S(x) as the interval of @’s for which 
C\(8) < x < C,(0). > cus 


Figure 1 
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By Section 2 of Chapter 4, the conditions of the lemma are satisfied in 
particular for a one-parameter exponential family, provided the tests are 
nonrandomized. In cases such as that of binomial or Poisson distributions, 
where the family is exponential but X is integer-valued so that randomiza- 
tion is required, the intervals can be obtained by applying the lemma to the 
variable X + V instead of X, where V is independent of X and uniformly 
distributed over (0, 1). 

In the binomial case, a table of the (randomized) uniformly most 
accurate unbiased confidence intervals is given by Blyth and Hutchinson 
(1960). The best choice of nonrandomized intervals and some large-sample 
approximations are discussed (and tables provided) by Blyth and Still 
(1983) and Blyth (1984). For additional discussion and references see 
Johnson and Kotz (1969, Section 3.7) and Ghosh (1979). 

In Lemma 2, the distribution of X was assumed to depend only on 0. 
Consider now the exponential family (1) in which nuisance parameters are 
present in addition to 6. The UMP unbiased tests of 0 = 6, are then 
performed as conditional tests given T = t, and the confidence intervals for 
0 will as a consequence also be obtained conditionally. If the conditional 
distributions are continuous, the acceptance regions will be of the form 


C,(6; t) < u < C,(8; t), 


where for each : the functions C; are increasing by Lemma 2. The 
confidence intervals are then 


~ Ci (ui t) &6 € Cou; t). 
If the conditional distributions are discrete, continuity can be obtained as 


before through addition of a uniform variable. 


: : ; ; ith 
Example 8. Poisson ratio. Let X and Y be independent Poisson variables wi 
means À and p, and let p = p/A. The conditional distribution of Y given X+Y=t 


is the binomial distribution b( p, t) with 
p 


PETING 


The UMP unbiased test $(y, £) of the hypothesis p = py is defined r each t as the 
UMP unbiased conditional test of the hypothesis p = po/(1 + po). I 


p(t) spp) 


are the associated most accurate unbiased confidence intervals for p given t, it 
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follows that the most accurate unbiased confidence intervals for »/A are 


O e, 39 
Lip (t) m A 2d Spey 


The binomial tests which determine the functions p(t) and p(t) are discussed in 
Example 1 of Chapter 4. 


8. REGRESSION 


The relation between two variables X and Y can be studied by drawing an 
unrestricted sample and observing the two variables for each subject, 
obtaining n pairs of measurements (Xj, Y,),...,(X,, Y,) (see Section 15 
and Chapter, 5, Problem 10). Alternatively, it is frequently possible to 
control one of the variables such as the age of a subject, the temperature at 
which an experiment is performed, or the strength of the treatment that is 
being applied. Observations Y,,..., Y, of Y can then be obtained at a 
number of predetermined levels x,,..., x, of x. Suppose that for fixed x 
the distribution of Y is normal with constant variance o? and a mean which 


is a function of x, the regression of Y on x, and which is assumed to be 
linear, 
E[Y|x] =a + Bx. 


If we put v; = (x, — x)/ VE(x, — x)" and y + ôv; = a + Bx; so that 
Xv, = 0, Lv? = 1, and 


x à 
a-i, 4- ; 
L(x- xy yxG; = x) 


the joint density of Y,,..., Y, is 


gyl- ally Bile 2 


These densities constitute an exponential family (1) with 


Umm aa ie pow, 
ô 1 

8-2— es Y 
02’ 9$, 2527 eim 
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This representation implies the existence of UMP unbiased tests of the 
hypotheses ay + bó =c where a, b, and c are given constants, and 
therefore of most accurate unbiased confidence intervals for the parameter 


p =ay bó. 


To obtain these ćonfidence intervals explicitly, one requires the UMP 
unbiased test of H : p = pọ, which is given by the acceptance region 


| JpEex + aF -ooa +? 
y/[za;- Fr- (Lox) -3 


G 


(45) 


where 


EOL 


(See Problem 33 and Chapter 7, Section 7, where there is also a discussion 
of the robustness of these procedures against nonnormality.) The resulting 
confidence intervals for p are centered at bLu,Y, + aY, and their length is 


- jÈ " E -YY-(Xenyo 


n-2 


It follows from the transformations given in Problem 33 that [X( Y; — 2 = 
(Zv,Y,)?]/o? has a x?-distribution with n — 2 degrees of freedom and hence 
that the expected length of the intervals is 


a? 2 
E(L) 2269] — +8. 


In particular applications, a and 5 typically are functions of the x's. If 
these are at the disposal of the experimenter and there is therefore some 
ed length of L is minimized by 


choice with respect to a and b, the expect ) 
minimizing (a?/n) + b?. Actually, it is not clear that the expected length is 
a good criterion for the accuracy of confidence intervals, since short 
intervals are desirable when they cover the true parameter value but not 
necessarily otherwise. However, the same result holds for other criteria 
Such as the expected value of (p — p) +(e T p) or more jene of 
(Ip — pl) + f.(lp — el), where f, and f; are increasing functions of their 


224 UNBIASEDNESS: APPLICATIONS; CONFIDENCE INTERVALS [5.8 


arguments. (See Problem 33.) Furthermore, the same choice of a and b also 
minimizes the probability of the intervals covering any false value of the 
parameter. We shall therefore consider (a?/n) + b? as an inverse measure 
of the accuracy of the intervals. 


E. le 9. Slope of regression line. Confidence levels for the slope 8 = 
8/ yX(x, - X)^ are obtained from the above intervals by letting a = 0 and b = 


1/ yX(x, — X)" . Here the accuracy increases with X(x; — x)’, and if the x, must 
be chosen from an interval [C,, C, ], it is maximized by putting half of the values at 
each end point. However, from a practical point of view, this is frequently not a 
good design, since it permits no check of the linearity of the regression. 


Example 10. Ordinate of regression line. Another parameter of interest is the 
value a + Bx, to be expected from an observation Y at x = Xo. Since 


8( x9 — x) 
Xx » x) 


a+ Bxo=y+ 


, 


the constants a and b are a— 1, b= (xo — X)/ yE( x, - xy. The maximum 


accuracy is obtained by minimizing |x — Xol and, if x = x, cannot be achieved 
exactly, also maximizing E(x, — x)?. 


Example 11. Intercept of regression line. Frequently it is of interest to estimate 
the point x at which a + Bx has a preassigned value. One may for example wish to 


find the dosage x = —a/B at which E(Y|x) = 0, or equivalently the value v = 
= =\2 : 
(x -= X)/ yE(x, — X) at which y + 8» = 0. Most accurate unbiased confidence 


sets for the solution —y/8 of this equation can be obtained from the UMP 
unbiased tests of the hypotheses —y/8 = vy. The acceptance regions of these tests 
are given by (45) with a = 1, b = v), and Po = 0, and the resulting confidence sets 
for v are the sets of values v satisfying 


= 1 
e[cs - Eoy] - 20Y(EnY) + — (C's? — nF?) > 0, 
A = 
where $° = [Y(Y, - Y) — (Zo, Y,)? ]/(n — 2). If the associated quadratic equation 


in v has roots v, v, the confidence statement becomes 


and 
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The somewhat surprising possibility that the confidence sets may be the outside 
of an interval actually is quite appropriate here. When the line y = y + àv is nearly 
parallel to the v-axis, the intercept with the v-axis will be large in absolute value, 
but its sign can be changed by a very small change in angle. There is the further 
possibility that the discriminant of the quadratic polynomial is negative, 


n¥? « (Eux) «cs, 


in which case the associated quadratic equation has no solutions. This condition 
implies that the leading coefficient of the quadratic polynomial is positive, so that 
the confidence set in this case becomes the whole real axis. The fact that the 
confidence sets are not necessarily finite intervals has led to the suggestion that their 
use be restricted to the cases in which they do have this form. Such usage will 
however affect the probability with which the sets cover the true value and hence the 
validity of the reported confidence coefficient.* 


9. BAYESIAN CONFIDENCE SETS 


The left side of the confidence statement (35) denotes the probability that 
the random set S( X) will contain the constant point 6. The interpretation 
of this probability statement, before X is observed, is clear: it refers to the 
frequency with which this random event will occur. Suppose for example 
that X is distributed as N(6, 1), and consider the confidence interval 


X —1.96 < 0 < X + 1.96 


corresponding to confidence coefficient y — .95. Then the random interval 
(X — 1.96, X + 1.96) will contain 0 with probability .95. Suppose now that 
X is observed to be 2.14. At this point, the earlier statement reduces to the 
inequality 0.18 « @ < 4.10, which no longer involves any random element. 
Since the only unknown quantity is 6, it is tempting (but not justified) to say 
that 0 lies between 0.18 and 4.10 with probability .95. ; 

To attach a meaningful probability to the event @ € S(x) when x is 
fixed requires that @ be random. Inferences made under the assumption that 
the parameter @ is itself a random (though unobservable) quantity with a 
known distribution are called Bayesian, and the distribution A of 0 before 
any observations are taken its prior distribution. After X — x has been 
observed, inferences concerning @ can be based on its conditional distribu- 
tion given x, the posterior distribution. In particular, any set S(x) with the 
Property 


P[ee s(x)|x =x] zv for all x 


*A method for obtaining the size of this effect was developed by Neyman, and tables have 
been computed on its basis by Fix. This work is reported by Bennett (1957). 
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is a 100% Bayesian confidence set or credible region for 0. In the rest of 
this section, the random variable with prior distribution A will be denoted 
by 6, with @ being the value taken on by © in the experiment at hand. 

Example 12. Normal mean. Suppose that © has a normal prior distribution 
N(, b?) and that given @ = 6, the variables X,,...,X,, are independent N(6, o?), 
9 known. Then the posterior distribution of 6 given x,,..., x, is normal with mean 
(Problem 34) 


nx/o? + p/b? 


eA) bd | ee 
Nx [8Ix] n/o? + 1/b 


and variance 


1 
r? = Valj] =. 
EAEI n/a? + 1/b7 


Since [9 — 9, ]/z, then has a standard normal distribution, the interval 7(x) with 
endpoints 


nx/o? + p/b? 1.96 


n/a? X 17g. * Vn/o? + 175 


satisfies P[® € I(x)|X = x] = .95 and is thus a 95% credible regi 
For n = 1, p = 0, o = 1, the interval reduces to a 


which for large b is Very close to the confidence i 
Np 4 ce interval fi t th 
beginning of the section. But now the statement that @ lies ies opimis 


obiil ueri aa AT A. 
pe A. 95 is justified, since it is a probability statement concerning the random 


assigning to @ the probability density 7(0) =c, -% « Ó < co. U 


resulting 7 is not a probabili density, si if thi 
fact is ignored and the resin diitcipeton df LI a Ceea Gen 
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Unlike confidence sets, Bayesian credible regions provide exactly the 
desired kind of probability statement even after the observations are known. 
They do so, however, at the cost of an additional assumption: that 0 is 
random and has a known prior distribution. Interpretations of such prior 
distributions as ways of utilizing past experience or as descriptions of a state 
of mind are discussed briefly in Chapter 4, Section 1 of TPE. Detailed 
accounts of the Bayesian approach and its application to credible regions 
can be found for example in Lindley (1965), Box and Tiao (1973), and 
Berger (1985); some frequency properties of such regions are discussed in 
Rubin (1984). The following examples provide a few illustrations and 
additional comments. 


Example 13. Let X be binomial b(p, n), and suppose that the prior distribu- 
tion for p is the beta distribution* B(a, b) with density Cp" !(1- p) ,0«p«1, 
0 « a, b. Then the posterior distribution of p given X — x is the beta distribution 
B(a + x,b+n-—x) (Problem 36). There are of course many sets S(x) whose 
probability under this distribution is equal to the prescribed coefficient y. A choice 
that is frequently recommended is the HPD (highest probability density) region, 
defined by the requirement that the posterior density of p given x be 2 k. 

With a beta prior, only the following possibilities can occur: for fixed x, 


(a) m(p|x) is decreasing, 

(b) (p|x) is increasing, 

(c) m(plx) is increasing in (0, po) and decreasing in ( po, 1) for some po, 

(d) m(p|x) is U-shaped, i.e. decreasing in (0, po) and increasing in ( po, 1) for 

some po. 

The HPD region then is of the form 

(à p< K(x), 

(b p> K(x), 

() K(x) <p < K(x), 

(d p< K,(x) or p > K(x), 
where the K’s are determined by the requirement that the posterior probability of 
un region, given x, be y; in cases (c) and (d) this condition must be supplemented 

y 


a[K,(x)|x] = LIESCSIBI 3 


4 


In general, if z(0|x) denotes the posterior density of 0, the HPD region is 
defined by 


n(O|x) =k 


"This is the so-called conjugate of the binomial distribution; for a more general discussion 
of conjugate distributions, see TPE, Chapter 4, Section 1. 
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with C determined by the size condition 


P[v(8|x) >k] = y: 


Example 14. Two-parameter normal: estimating the mean. Let Kis- s1 Xa be 
independent N(¢, o?), and for the sake of simplicity suppose that (£, o) has the 
joint improper prior density given by 


1 
7(,0) dí do = df= do forall -0 «£« o, 0«o, 


which is frequently used to model absence of information concerning the parame- 
ters. Then the joint posterior density of (£, o) given x = (5... x,) is of the form 


1 d ug 
T(E, olx) déde = C(x) rre] - 5 Y (£— x)? | dt do. 
i=l 


Determination of a credible region for £ requires the marginal posterior density of £ 
given x, which is obtained by integrating the joint posterior density with respect to 
o. These densities depend only on the sufficient statistics x and S? = D(x, — x)?, 
and the posterior density of £ is of the form (Problem 37) 


n/2 


1 
A(x)| ——————— 
(x) "UPS 
S2 


1+ 


Here X and S enter only as location and scale parameters, and the linear function 


Yn(E- 3) 
sha =T 


of £ has the r-distribution with n — 1 d i i i 
distribution of £ for fixed £ and c bes DU li ages wit tc 


A ven in Section 2, the credible 100(1 — a) 


t 


Yn(£- x) 
sin —1i 


is formally identical with the confidence intervals 
under different assumptions, and their interpre 


RE 


s (37). However, they are derived 
tation differs accordingly. 
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Example 15. Two-parameter normal: estimating o. Under the assumptions of 
the preceding example, credible regions for a are based on the posterior distribution 
of.c given x, obtained by integrating the joint posterior density of (£, 0) with 
respect to £. Using the fact that Y(£ — x,)? = n(£ — x)? + L(x, — x)’, it is seen 
(Problem 38) that given x, the conditional (posterior) distribution of D(x, — x)?/o? 
is x? with n — 1 degrees of freedom. As in the case of the mean, this agrees with the 
sampling distribution of the same quantity when o is a (constant) parameter, given 
in Section 2. (The agreement in both cases of two distributions derived under such 
different assumptions is a consequence of the particular choice of the prior distribu- 
tion and the fact that it is invariant in the sense of TPE, Section 4.4.) A change of 
variables now gives the posterior density of o and shows that 7(o|x) is of the form 
(c) of Example 13, so that the HPD region is of the form K,(x) < o < K;(x) with 
0 « K,(x) < K;(x) < oo. 

Suppose that a credible region is required, not for o, but for o for some r > 0. 
For consistency, this should then be given by [K,(x)]’ < o” < [K,(x)]’, but this is 
not the case, since the relative height of the density of a random variable at two 
points is not invariant under monotone transformations of the variable. In fact, in 
the present case, the HPD region for o^ will become one-sided for sufficiently large 
r although it is two-sided for r = 1 (Problem 38). 


Such inconsistencies do not occur if the HPD region is replaced by ` 
the equal-tails interval (C,(x), C,(x)) for which P[O < C,(x)| X = x] = 
P[O > C,(x)| X = x] = (1 — y)/2.* More generally inconsistencies under 
transformations of © are avoided when the posterior distribution of © is 
summarized by a number of its percentiles corresponding to the standard 
confidence points mentioned in Chapter 3, Section 5. Such a set is a 
compromise between providing the complete posterior distribution and 
providing a single interval corresponding to only two percentiles. 

Both the confidence and the Bayes approach present difficulties: the first, 
the problem of postdata interpretation; the second, the choice of a prior 
distribution and the interpretation of the posterior coverage probabilities if 
there is no clear basis for this choice. It is therefore not surprising that 
efforts have been made to find an approach without these drawbacks. The 
first such attempt, from which most later ones derive, is due to Fisher [1930; 
for his final account see Fisher (1973)]. : 

To discuss Fisher's concept of fiducial probability, consider once more 
the example at the beginning of the section, in which X is distributed as 
N(0,1). Since then X — 6 is distributed as N(0, 1), so is 0 — X, and hence 


P(0—Xxy)-9(y) forall y. 
For fixed X = x, this is the formal statement that a random variable 0 has 
distribution N(x, 1). Without assuming @ to be random, Fisher calls N(x, 1) 
the fiducial distribution of 0. Since this distribution is to embody the 


"They also do not occur when the posterior distribution of @ is discrete. 
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information about 0 provided by the data, it should be unique, and Fisher 
imposes conditions which he hopes will ensure uniqueness. This leads to 
some technical difficulties, but more basic is the question of how to interpret 
fiducial probability. In a series of independent repetitions of the experiment 
with arbitrarily varying 6,, the quantities bi — X 6, — X,,... will con- 
stitute a sequence of independent standard normal variables. From this fact, 
Fisher attempts to derive the fiducial distribution N(x, 1) of @ as a 
frequency distribution with respect to an appropriate reference set. How- 
ever, this argument is difficult to follow and unconvincing. For summaries 
of the fiducial literature and of later related developments by Dempster, 
Fraser, and others, see Pedersen (1978), Buehler (1980), Dawid and Stone 
(1982), and the encyclopedia articles by Fraser (1978), Edwards (1983), 
Buehler (1983), and Stone (1983). 

Fisher’s effort to define a suitable frame of reference led him to the 
important concept of relevant subsets, which will be discussed in Chapter 10. 


10. PERMUTATION TESTS 


For the comparison of a treatment with a control situation in which no 
treatment is given, it was shown in Section 3 that the one-sided t-test is 
UMP unbiased for testing H:=£ against ņn — £— A > 0 when the 
measurements X;,..., Xm and Y,,..., Y, are samples from normal popula- 
. tions NG, 0?) and N(, o?). It was further shown in Section 4 that the level 
of this test is (asymptotically) robust against nonnormality—that is, that 
except for small m or n the level of the test is approximately equal to the 
nominal level a when the X’s and Y’s are samples from any distributions 
with densities f(x) and f(y — A) with finite variance. If such an approxi- 
mate level is not satisfactory, one may prefer to try to obtain an exact 
level-a unbiased test (valid for all f) by replacing the original normal model 
with the nonparametric model for which the joint density of the variables is 


(49) 7 FG 2 SEDI ENLOF fes. 


pi biles take F to be the family of all probability densities that are 

If there is much variation in the population being sampled, the sensitivity 
of the experiment can frequently be increased by dividing the population 
into more homogeneous subgroups, defined for example by some character- 
istic such as age or sex. A sample of size N, (i = 1,..., c) is then taken from 
the ith subpopulation: m, to serve as controls, and the other n; = N; - Mi 
to receive the treatment. If the observations in the ith subgroup of such a 
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stratified sample are denoted by 


(X555 X Yaraa) = (Zine Zin.) 


the density of Z = (Z,,,..., Zen,) is 
(47) pala) = THA Ca) + Gina = 8) fus, - &)]. 


Unbiasedness of a test $ for testing A = 0 against A > 0 implies that for all 
fis fe 


(48) [o(2)po(z) & = a (dz = dz... dz... 


Theorem 6. If F is the family of all probability densities f that are 
continuous a.e., then (48) holds for all f,,..., f. € F if and only if 


1 
(49) —— Y 9(z7)2-a ae, 
XR OL 


where S(z) is the set of points obtained from z by permuting for each 
i = L,...,c the coordinates z;; (j =1,..., N,) within the ith subgroup in all 
N,!... N! possible ways. 

Proof. To prove the result for the case c = 1, note that the set of order 
Statistics T(Z) = (Zq),..-, Zi) is a complete sufficient Statistic for F 
(Chapter 4, Example 6). A necessary and sufficient condition for (48) is 
therefore 


(50) E[e«(Z)]T(2]-« ae. 


The set S(z) in the present case (c — 1) consists of the N! points 
obtained from z through permutation of coordinates, so that S(z) = 
(z': T(z’) = T(z)). It follows from Section 4 of Chapter 2 that the condi- 
tional distribution of Z given T(z) assigns probability 1/N ! to each of the 
N! points of S(z). Thus (50) is equivalent to 
Ei Y e(7)-a ae, 


N! z'eS(z) 


(51) 


as was to be proved. The proof for general c is completely analogous and is 
left as an exercise (Problem 44.) 
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The tests satisfying (49) are called permutation tests. An extension of this 
definition is given in Problem 54. 


11. MOST POWERFUL PERMUTATION TESTS 


For the problem of testing the hypothesis H : A = 0 of no treatment effect 
on the basis of a stratified sample with density (47) it was shown in the 
preceding section that unbiasedness implies (49). We shall now determine 
the test which, subject to (49), maximizes the power against a fixed alterna- 
tive (47) or more generally against an alternative with arbitrary fixed density 
h(z). 

The power of a test $ against an alternative h is 


COLO) d= JEI] dP7(t). 


Let t = T(z) = (zq),---, z(w;), so that S(z) = S(t). As was seen in Exam- 
ple 7 and Problem 5 of Chapter 2, the conditional expectation of $(Z) 
given T(Z) = t is 


Y 9(z)h(z) 


= ZES) 
y(t) ZOR 


z€S(t) 


To maximize the power of subject to (49) it is therefore necessary to 
maximize y(t) for each t subject to this condition. The problem thus 
reduces to the determination of a function $ which, subject to 


1 
X e(2—— —— - a, 
ae A 


maximizes 
h(z) 
QUO X h(z) 
z'eS() 


By the Neyman-Pearson fundamental lemma, this is achi jecti 
- a, this is achieved by rejectin 
H for those points z of S(r) for which the ratio MULT 
h(z)N,... NJ 
Y A(z’) 


z'es(n 
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is too large. Thus the most powerful test is given by the critical function 


1 when A(z) > C[T(z)], 
(52) $(z)={y when A(z) = C[r(z)], 
0 when A(z) <C[T7(z)]. 


To carry out the test, the N,!...N,! points of each set S(z) are ordered 
according to the values of the density A. The hypothesis is rejected for the k 
largest values and with probability y for the (k + 1)st value, where k and y 
are defined by 


k4 y 2 aM... NI. 


Consider now in particular the alternatives (47). The most powerful permu- 
tation test is seen to depend on A and the f,, and is therefore not UMP. 

Of special interest is the class of normal alternatives with common 
variance: 


f= N( » 9). 


The most powerful test against these alternatives, which turns out to be 
independent of the £,, 7, and A, is appropriate when approximate normal- 
ity is suspected but the assumption is not felt to be reliable. It may then be 
desirable to control the size of the test at level a regardless of the form of 
the densities f, and to have the test unbiased against all alternatives (47). 
However, among the class of tests satisfying these broad restrictions it is 
natural to make the selection so as to maximize the power against the type 
of alternative one expects to encounter, that is, against the normal alterna- 
tives. 
With the above choice of f;, (47) becomes 


i=1\j=1 


(53) -A(z) = (Zo) "o-z } | Y (z;- &) 
N 2 
+ X (gyi A) || 
j=m;+1 


Since the factor exp[ —L,L%,(z;; — §;)?/207] is constant over S(t), the test 
(52) therefore rejects H when exp(AE,EM.,, az) > C[T(z)] and hence 
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when 


c 


c ni N; 
(54) x E= p z> C[T(z)]. 


i=1 j=1 i=1 j=m;+1 


Of the N,!... N,! values that the test statistic takes on over S(t), only 


N, N, 

ea es 
are distinct, since the value of the statistic is the same for any two points z' 
and z” for which (Zj, ..., Z/m,) and (z5,..., Z/m,) are permutations of each 
other for each i. It is therefore enough to compare these distinct values, and 
to reject H for the k’ largest ones and with probability y’ for the (k' + 1)st, 


where 
k' vial c 
(m)--(%). 


The test (54) is most powerful against the normal alternatives under 
consideration among all tests which are unbiased and of level a for testing 
H: A = 0 in the original family (47) with f,,..., f, € ¥.* To complete the 
proof of this statement it is still necessary to prove the test unbiased against 
the alternatives (47). We shall show more generally that it is unbiased 
against all alternatives for which X; (f= 1,...,m,), Y, (k =1,.--, 1) 
are independently distributed with cumulative distribution functions F,, G; 
respectively such that Yj, is stochastically larger than X,,, that is, such that 
G,(z) < F(z) for all z. This is a consequence of the following lemma. 


Lemma Sa Ghat: Xi easy X, Yos ovs Y, be samples from continuous distri- 
butions F, G, and let $(x1,..., Xm yy. .., Yq) bea critical function such that 
(a) its expectation is a whenever G = F, and (b) y< y for i 1,..., n 
implies B i 


Ope tens yu CASES $(xy S exusta Yi). 
Then the expectation B = B(F, G) of $ is > a for all pai istributi 
í e i B pairs of distributions 
for which Y is stochastically larger than X; it is < a if X is stochastically 
larger than Y. 


Proof. By Lemma 1 of Chapter 3 there exist functions f. g and inde- 
pendent random variables V,,...., V,,,,, such that the distributions of IV) 


*For a closely related result, see Odén and Wedel (1975). 
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and g(V,) are F and G respectively and that f(z) < g(z) for all z. Then 


E| AV) -s Vn); Vm): Vms] = a 


and 
E¢[f(Vi)»-+->f(Vm)s Vardir 8(9544)] = 8. 
Since for all (v... Um+n) 
ef)... fs): f Cis f(Om+n)] 
< é[f(v,)..... f(m); (02) B(Umen)] 


the same inequality holds for the expectations of both sides, and hence 
a x p. 

The proof for the case that X is stochastically larger than Y is completely 
analogous. 

The lemma also generalizes to the case of c vectors (X1,..-, Xim, 
Y,,..., Yj,,) with distributions (F, G,). If the expectation of a function $ is 
then a when F, = G, and ¢ is nondecreasing in each y;; when all other 
variables are held fixed, it follows as before that the expectation of $ is > a 
when the random variables with distribution G; are stochastically larger 
than those with distribution F;. 

In applying the lemma to the permutation test (54) it is enough to 
consider the case c=1, the argument in the more general case being 
completely analogous. Since the rejection probability of the test (54) is a 
whenever F = G, it is only necessary to show that the critical function of 
the test satisfies (b). Now $ = 1 if 7777, ,z, exceeds sufficiently many of the 


sums 17777, ,2,, and hence if sufficiently many of the differences 
m+n m+n 
» zT a Z; 
i=m+1 i=m+1 


Laza- E on-XEun-XEs 
i=m+1 i-mtl i-1 i= 
where r, < --+ < r, denote those of the integers jm+1--+» Jp 4, that are 
« s. those of the integers m t L,...,m * n not 


<m, and s< | <s, 5 3r ; 
included in the set (jst +> Jn) If Zz,, — Ezr is positive and y, < yf, 
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that is, z; < z? fori ^ m + 1,..., m + n, then the difference Dz = Ez, is 
also positive and hence ¢ satisfies (b). 

The same argument also shows that the rejection probability of the test is 
< a when the density of the variables is given by (47) with A < 0. The test 
is therefore equally appropriate if the hypothesis A = 0 is replaced by 
A<0. 

Except for small values of the sample sizes N,, the amount of computa- 
tion required to carry out the permutation test (54) is very large. Computa- 
tional methods are discussed by Green (1977) and John and Robinson 
(1983b). Alternatively, several large-sample approximations for the critical 
value are available; see, for example, Robinson (1982). 

A particularly simple approximation relates the permutation test to the 
corresponding t-test. On multiplying both sides of the inequality 


X» C[T(z)] 


by (1/m) + (1/n) and subtracting (Cx, + Ly,)/m, the rejection region for 
c=1 becomes j-X» C[T(2)] or W- (y- X) Vx, - z) > 
C[T(z)) since the denominator of W is constant over S(z) and hence 
depends only on 7(z). As was seen at the end of Section 3, this is equivalent 


to 
EG-xrEG-» era» o0 


The rejection region therefore has the form of a t-test in which the constant 
cutoff point C, of (27) has been replaced by a random one. It turns out that 
when the hypothesis is true, so that the Z’s are identically and indepen- 
dently distributed, and if E|Z|? < co and m/n is bounded away from zero 
and infinity as m and n tend to infinity, the difference between the random 
cutoff point C[T(Z)] and C, tends to zero in probability. In the limit, the 
Permutation test therefore becomes equivalent to the t-test given by 
(27)-(29).* It follows that the permutation test can be approximated for large 
samples by the standard t-test. Exactly analogous results hold for c > 1; the 
appropriate t-test is provided in Chapter 7, Problem 9. 


(55) 


"This equivalence is not limited to the behavio i i 
3 ~ 1 r under the hypothesis. For large samples, it 
is shown by Hoeffding (1952) and Bickel and van Zwet (1978, Theorem 7.2) that esd D 


of the permutation test is approximate] Tas 
further references see oun q99. eee e io a 
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The problem of testing for the effect of a treatment was considered in 
Section 3 under the assumption that the treatment and control measure- 
ments X;,..., Xm and Y,,..., Y, constitute samples from normal distribu- 
tions, and in Sections 10 and 11 without relying on the assumption of 
normality. We shall now consider in somewhat more detail the structure of 
the experiment from which the data are obtained, resuming for the moment 
the assumption that the distributions involved are normal. 

Suppose that the experimental material consists of m +n patients, 
plants, pieces of material, or the like, drawn at random from the population 
to which the treatment could be applied. The treatment is given to n of 
these while the other m serve as controls. The characteristic that is to be 
influenced by the treatment is then measured in each case, leading to 
Observations X,,..., Xm Yy,..., Yp 

To be specific, suppose that the treatment is carried out by injecting a 
drug and that m + n ampules are assigned to the m + n patients. The ith 
measurement can be considered as the sum of two components. One, say U,, 
is associated with the ith patient; the other, V,, with the ith ampule and the 
circumstances under which it is administered and under which the measure- 
ments are taken. The variables U, and V, are assumed to be independently 
distributed, the V's with normal distribution N(n, o?) or N(£, o?) as the 
ampule contains the drug or is one of those used for control. If in addition 
the U's are assumed to constitute a random sample from N(p, of), it 
follows that the X's and Y's are independently normally distributed with 
common variance o? + o? and means 


E(X)-u*&  E(Y)-n*nm 


Except for a change of notation their joint distribution is then given by (26), 
and the hypothesis ņ = £ can be tested by the standard t-test. 
Unfortunately, under actual experimental conditions, it is frequently not 
possible to ensure that the patients or other experimental units constitute a 
random sample from the population of such units. They may be patients in 
a certain hospital at a given time, or volunteers for an experiment, and may 
constitute a haphazard rather than a random sample. In this case the U’s 
would have to be considered as unknown constants, since they are not 
Obtained by any definite sampling procedure. This assumption is ap- 
Propriate also in a different context. Suppose that the experimental units are 
all the machines in a shop or fields on a farm. If the experiment is 
Performed only to determine the best method for this particular shop or 
farm, these experimental units are the only relevant ones; that is, a repli- 
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cation of the experiment would consist in comparing the two treatments 
again for the same machines or fields rather than for a new batch drawn at 
random from a large population. In this case the units themselves, and 
therefore the u’s, are constant. 

Under the above assumptions the joint density of the m + n measure- 
ments is 


1 1 - a 
germ- mle -u — $) + EGO XUL TUE »l 
Since the u's are completely arbitrary, it is clearly impossible to distinguish 
between H : ņ = £ and the alternatives K : n > £. In fact, every distribution 
of K also belongs to H and vice versa, and the most powerful level-a test 
for testing H against any simple alternative specifying £, 7, o, and the u's 
rejects H with probability a regardless of the observations. 

Data which could serve as a basis for testing whether or not the 
treatment has an effect can be obtained through the fundamental device of 
randomization. Suppose that the N = m + n patients are assigned to the N 
ampules at random, that is, in such a way that each of the N! possible 
assignments has probability 1/N! of being chosen. Then for a given 
assignment the N measurements are independently normally distributed 
with variance o? and means £ + u, (i=1,...,m) and ņ + u, (i=m+ 
1,..., m * n). The overall joint density of the variables a 


(Zeta ur votre els du e Y,) 
is therefore 
1 1 


VE 


(56) ERN 
* Cheeses dw) (Vro) ji 


1 m n 
9 MR EA 22 
e sat Ets K 9 a EG Den » 
where the outer summation extends over all N! i j j 
r ! permutations (j,,..., jy) of 
(1,..., N). Under the hypothesis n = £ this density can be iten as ¥ 
1 


amus ji TT 
C7 N! Pol marl as & (4 T 21 


where §, = 4, += uy +n. 
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Without randomization, a set of y’s which is large relative to the x-values 
could be explained entirely in terms of the unit effects u;. However, if these 
are assigned to the y’s at random, they will on the average balance those 
assigned to the x’s. As a consequence, a marked superiority of the second 
sample becomes very unlikely under the hypothesis, and must therefore be 
attributed to the effectiveness of the treatment. 

The method of assigning the treatments to the experimental units com- 
pletely at random permits the construction of a level-a test of the hypothesis 
n = £, whose power exceeds a against all alternatives ņn — > 0. The actual 
power of such a test will however depend not only on the alternative value 
of n — £, which measures the effect of the treatment, but also on the unit 
effects u,. In particular, if there is excessive variation among the u’s, this 
will swamp the treatment effect (much in the same way as an increase in the 
variance o? would), and the test will accordingly have little power to detect 
any given alternative n — £. 

In such cases the sensitivity of the experiment can be increased by an 
approach exactly analogous to the method of stratified sampling discussed 
in Section 10. In the present case this means replacing the process of 
complete randomization described above by a more restricted randomiza- 
tion procedure. The experimental material is divided into subgroups, which 
are more homogeneous than the material as a whole, so that within each 
group the differences among the u's are small. In animal experiments, for 
example, this can frequently be achieved by a division into litters. Random- 
ization is then applied only within each group. If the ith group contains N, 
units, n; of these are selected at random to receive the treatment, and the 
remaining m, = N, — n, serve as controls (LN, = N, Xm, = m, Yn, =n). 

An example of this approach is the method of matched pairs. Here the 
experimental units are divided into pairs, which are as like each other as 
Possible with respect to all relevant properties, so that within each pair the 
difference of the u's will be as small as possible. Suppose that the material 
consists of n such pairs, and denote the associated unit effects (the U's of 
the previous discussion) by Uj, Uf; ...; Up, Uj. Let the first and second 
member of each pair receive the treatment or serve as control respectively, 
and let the observations for the ith pair be X, and Y,. If the matching is 
completely successful, as may be the cáse, for example, when the same 
patient is used twice in the investigation of a sleeping drug, or when 
identical twins are used, then U; = U, for all i, and the density of the X’s 
and Y's is 


1 1 2 MERO E 
6) ca osee|- lE tcn Eoo 4] 
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The UMP unbiased test for testing H: ņ = £ against n > £ is then given in 
terms of the differences W, = Y, — X, by the rejection region 


(59) ins — Ew- w) >C. 


(See Problem 48.) 

However, usually one is not willing to trust the assumption u; = u; even 
after matching, and it again becomes necessary to randomize. Since as a 
result of the matching the variability of the u’s within each pair is 
presumably considerably smaller than the overall variation, randomization 
is carried out only within each pair. For each pair, one of the units is 
selected with probability 1 to receive the treatment, while the other serves as 
control. The density of the X’s and Y’s is then 


Ld 1 
(60) TH {exe|- zal - £7 O-n- w] 


sapere 
2^ (Vro) "i-i | 


1 
to-le -£- ul t (y-n- lle 
Under the hypothesis ņ = £, and writing 
Za 7X5 Z37J» ia76tu, Santu. (i=1,...,7), 
this becomes 
1 1 li om x 
(61) =) exp] - — JE 
Be way wEiU 
Here the outer summation extends over the 2” poi ' 2 (oY * 
Tee QUIETE points i = (£f, ..., $72) for 
which (51, 12) is either (£j, E) or (Ea. ba) 5 lied 
13. PERMUTATION TESTS AND RANDOMIZATION 


It was shown in the preceding section that randomization provides a basis 
for testing the hypothesis ņ = £ of no treatment effect, without any assump- 
tions concerning the experimental units. In the present section, a specific 
test will be derived for this problem. When the experimental units are 
treated as constants, the probability density of the observations is given by 
(56) in the case of complete randomization and by (60) in the case of 
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matched pairs. More generally, let the experimental material be divided into 
c subgroups, let the randomization be applied within each subgroup, and let 
the observations in the ith subgroup be 


(Zo Zin,) 5 (X... Xim? Yay Y: 


For any point u = (u,,..., Uey.) let S(u) denote as before the set of 
N,!...N.! points obtained from u by permuting the coordinates within 
each subgroup in all N,!... N,! possible ways. Then the joint density of the 
Z's given u is 

1 1 
NIGUNS Viene 
N,!... N! wesu) (V2mo) 


Xexp| — => 
2e? 7 j=l j=m;+1 


c |m N, 
25 | » (zy x43 uj) * ^» (25 - 7-— 2 


and under the hypothesis of no treatment effect 
(63) 


G)- — b SE ea 
(2) = —— exp] - = 2, — $) | 
Po. NJ... NJ pézo Wro)” er 1 


It may happen that the coordinates of u or $ are not distinct. If then 
some of the points of S(u) or S(§) also coincide, each should be counted 
with its proper multiplicity. More precisely, if the N,!...N,! relevant 
permutations of N,---: +N, coordinates are denoted by gy k - 
1,..., N,!... N,!, then S(£) can be taken to be the ordered set of points 
gib, k =1,..., N,!... N,!, and (63), for example, becomes 


1 M! NA 


As alt j 
veal) mee qm caer id 


where |u|? stands for Df_,54 uj). 


Theorem 7. A necessary and sufficient condition for a critical function > 
to satisfy 


(64) fep. sa (d= dij -.. deen.) 
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for all o > 0 and all vectors $ is that 


(65) ; Y é(z)ze' “ace. 


NEN ee 


The proof will be based on the following lemma. 


Lemma 4. Let A be a set in N-space with positive Lebesgue measure 
ICA). Then for any € > 0 there exist real numbers a > 0 and §,,..., £y such 
that 


PME TO 


where the X's are independently normally distributed with means E( X,) = &; 
and variance 0% = 0°. 


Proof. Suppose without loss of generality that (A) < oo. Given any 
n > 0, there exists a square Q such that 


u(Q A) < m(Q). 


This follows from the fact that almost every point of A has metric density 
1 or from the more elementary fact that a measurable set can be ap- 
proximated in measure by unions of disjoint squares. Let a be such that 


| e i €\I/N 
a: -$Ja- EET. 


and let 


If (5,,..., £y) is the center of Q, and if o = b/a = (1/2a 1/N where 
2b is the length of the side of Q, then ` Ee 


il 
TT hinl- Ei m 2 dx,...dxy 
1 1 
* asy hovel za Eos e| anas 


m HER a)’ - 2 


*See for example Hobson (1927). 
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On the other hand, 


md ad ED E) | dx, ... dry 


1 E. € 
< (aso) M 0 9) < E 


and by adding the two inequalities one obtains the desired result. 
Proof of the theorem. Let be any critical function, and let 


W)= x E 69 


LUN Tess) 


If (65) does not hold, there exists n > 0 such that Y(z) > a + n on a set A 
of positive measure. By the Lemma there exists ø > 0 and $ = (5... fen) 
such that P(Z € A} > 1 — n when Z,..., Zey, are independently nor- 
mally distributed with common variance o? and means E(Z,;) = y It 
follows that 


(66) 


Jol)pa.rlz) de = fylz)Pa, (z) d 
1 


1 2 
> [A Gamay á uL YXG-&)|4 


» (a4 9)(1 - 2), 


which is » a, since a + 7 < 1. This proves that (64) implies (65). The 
converse follows from the first equality in (66). 


Corollary 3. Let H be the class of densities 
{ Po,¢(z): 0 > 0, 70 < (ues oo). 


A complete family of tests for H at level of significance a is the class of tests € 
satisfying 


1 
(67) A Nale mas ae. 
ET 
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Proof. The corollary states that for any given level-a test $, there exists 
an element $ of @ which is uniformly at least as powerful as ¢). By the 
preceding theorem the average value of $, over each set S(z) is < o. On 
the sets for which this inequality is strict, one can increase $, to obtain a 
critical function $ satisfying (67), and such that $)(z) x $(z) for all z. 
Since against all alternatives the power of ¢ is at least that of 99, this 
establishes the result. An explicit construction of $, which shows that it can 
be chosen to be measurable, is given in Problem 51. 

This corollary shows that the normal randomization model (62) leads 
exactly to the class of tests that was previously found to be relevant when 
the U's constituted a sample but the assumption of normality was not 
imposed. It therefore follows from Section 11 that the most powerful level-a 
test for testing (63) against a simple alternative (62) is given by (52) with 
h(z) equal to the probability density (62). If  — £ = A, the rejection region 
of this test reduces to 


ds 2 N N, 
(688 en E [Essa 2002, — un) es C[T(z)], 
u'€S(u) +1 


9 jeje j=m, 


since both Lz, and LYz?, are constant on S(z) and therefore functions 
only of T(z). It is seen that this test depends on A and the unit effects u 
so that a UMP test does not exist. 

Among the alternatives (62) a subclass occupies a central position and is 
of particular interest. This is the class of alternatives specified by the 
assumption that the unit effects u, constitute a sample from a normal 
distribution. Although this assumption cannot be expected to hold 
exactly—in fact, it was just as a safeguard against the possibility of its 
breakdown that randomization was introduced—it is in many cases reason- 
able to suppose that it holds at least approximately. The resulting subclass 
of alternatives is given by the probability densities 


ij? 


1 
(69) Uy 


1L mi N, 
sos BORET T (eu-u-]l 
j=m;+ 


_ These alternatives are suggestive also from a slightly different point of 
view. The procedure of assigning the experimental units to the treatments at 
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random within each subgroup was seen to be appropriate when the varia- 
tion of the u's is small within these groups and is employed when this is 
believed to be the case. This suggests, at least as an approximation, the 
assumption of constant u;, = u; which is the limiting case of a normal 
distribution as the variance tends to zero, and for which the density is also 
given by (69). 

Since the alternatives (69) are the same as the alternatives (53) of Section 
11 with u, — £ = £, u; — n = £j — A, the permutation test (54) is seen to be 
most powerful for testing the hypothesis n = & in the normal randomization 
model (62) against the alternatives (69) with n — £ > 0. The test retains this 
property in the still more general setting in which neither normality nor the 
sample property of the U's is assumed to hold. Let the joint density of the 
variables be 


€ mi N; 
(70) 25 II A(z, MRSS é) ) IT AC Te 1) |. 
j=m;+ 


wes(u)i=l|j=1 


with f, continuous a.e. but otherwise unspecified.* Under the hypothesis 
H :» = &, this density is symmetric in the variables (25, ..., 2;y,) of the ith 
subgroup for each i, so that any permutation test (49) has rejection 
probability a for all distributions of H. By Corollary 3, these permutation 
tests therefore constitute a complete class, and the result follows. 


14. RANDOMIZATION MODEL AND CONFIDENCE 
INTERVALS 


In the preceding section, the unit responses u; were unknown constants 
(parameters) which were observed with error, the latter represented by the 
random terms V,. A limiting case assumes that the variation of the V''s is so 
small compared with that of the u's that these error variables can be taken 
to be constant, i.e. that V, = v. The constant v can then be absorbed into 
the u's, and can therefore be assumed to be zero. This leads to the following 
two-sample randomization model: - 

N subjects would give “true” responses u, .... uy if used as controls. 
The subjects are assigned at random, n to treatment and m to control. If 
the responses are denoted by X,,..., Xm and Y,,..., Y, as before, then 
under the hypothesis H of no treatment effect, the X s and Y's are a 
random permutation of the u's. Under this model, in which the random 


"Actually, all that is needed is that fy,....fc € F, where F is any family containing all 
normal distributions. 
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assignment of the subjects to treatment and control constitutes the only 
random element, the probability of the rejection region (55) is the same as 
under the more elaborate models of the preceding sections. 

. The corresponding limiting model under the alternatives assumes that the 
treatment has the effect of adding a constant amount A to the unit response, 
so that the X’s and Y's are given by (u,,...,u; 5 uj, + A,..., ui, + A) 
for some parmutation (i, ..., iy) of (1,..., N). 

These models generalize in the obvious way to stratified samples. In 
particular, for paired comparisons it is assumed under H that the unit 
effects (uj, u/) are constants, of which one is assigned at random to 
treatment and the other to control. Thus the pair ( X;, Y;) is equal to (u;, u;) 
or (u’, u;) with probability + each, and the assignments in the n pairs are 
independent; the sample space consists of 2" points each of which has 
probability (3)". Under the alternative, it is assumed as before that A is 
added to each treated subject, so that P(X, = u; Y, = u; + A) = P(X; = uj, 
Y, = u; + A) = 1. The distribution generated for the observations by such a 
randomization model is exactly the conditional distribution given T(z) of 
the preceding sections. In the two-sample case, for example, this common 
distribution is specified by the fact that all permutations of (X;,.... X,; 
Y, — A,..., Y, — A) are equally likely. As a consequence, the power of the 
test (55) in the randomization model is also the conditional power in the 
two-sample model (46). As was pointed out in Chapter 4, Section 4, the 
conditional power B(A|T(z)) can be interpreted as an unbiased estimate of 
the unconditional power 8,-(A) in the two-sample model. The advantage of 
BC A|T(z)) is that it depends only on A, not on the unknown F. Approxima- 
tions to B(A|T(z)) are discussed by Robinson (1973, 1982), John and 
Robinson (1983a), and Gabriel and Hsu (1983). 

The tests (54), which apply to all three models—the sampling model (47), 
the randomization model, and the intermediate model (70)—can be inverted 
in the usual way to produce confidence sets for A. We shall now determine 
these sets explicitly for the paired comparisons and the two-sample case. 
The derivations will be carried out in the randomization model. However, 
they apply equally in the other two models, since the tests, and therefore the 
associated confidence sets, are identical for the three models. 

Consider first the case of paired observations (xj, yj) i 9 1,...,n. The 
one-sided test rejects H : A = 0 in favor of A > 0 when Y^ Ui i among the 
K largest of the 2" sums obtained by replacing y, by x, for all, some, or 
none of the values i = 1,..., n. (It is assumed here for he sake of simplicity 
that a = K/2", so that the test requires no randomization to achieve the 
ky level a.) Re d, = y, — x, = 2y, — ty, where t, = x, + y, is fixed. Then 

e test 1s equivalent to rejecting when Yd; is one of the K largest of the 2" 
values È + d;, since an interchange of y; with x, is equivalent to replacing 
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d; by —d,. Consider now testing H: A = A, against A > Ao. The test then 
accepts when Y(d; — Ap) is one of the / = 2" — K smallest of the 2” sums 
Y + (d; — Ag), since it is now y; — A, that is being interchanged with x;. 
We shall next invert this statement, replacing A, by A, and see that it is 
equivalent to a lower confidence bound for A. 

In the inequality 


(71) X(4,- 4) < X[&(4; - )]. 


suppose that on the right side the minus sign attaches to the (d; — A) with 
i=i,,...,i, and the plus sign to the remaining terms. Then (71) is 
equivalent to 


dyte sito da AO Sa OC ee A 


h 


Thus, X(d, — A) is among the / smallest of the Y + (d; — A) if and only if 
at least 2" — / of the M = 2" — 1 averages (d; + `+: +d, )/r are < Ay ie. 
if and only if 8x, < A, where à < -** < ôm) is the ordered set of 
averages (d, + --- +d; )/r, r = 1,..., M. This establishes ôx) as a lower 
confidence bound for A at confidence level y = K/2". [Among all con- 
fidence sets that are unbiased in the model (47) with m; — n; — 1 and 
c= n, these bounds minimize the probability of falling below any value 
N < A for the normal model (53).] } 

By putting successively K = 1,2,...,2", it is seen that the M + 1 inter- 
vals 


(72) (— œ, 83), (89:99). (50 êm)» (êm ©) 


each have probability 1/(M + 1) = 1/2" of containing the unknown, A. 
The two-sided confidence intervals (6x), 9») with y = (2" - 
K)/2"-! correspond to the two-sided version of the test (54) with error 
probability (1 — y)/2 in each tail. A suitable subset of the points 
1)» - - Ô) constitutes a set of confidence points in the sense of Chapter 3, 
Section 5. 

The inversion procedure for the two-group case is quite analogous. Let 
Qn... Xm Yis- -3 Yq) denote the m control and n treatment observations, 
and suppose without loss of generality that m <n. Then the hypothesis 
A = A, is accepted against A > Ao if 17..,(y; — Ao) is among the / smallest 
of the (m E n eee obtained by replacing a subset of the (y; — Ao)’s with 
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x's. The inequality 
X(y-&) cs (x; fus ag *x,) T [» SECUS e (n - r)A], 


with (i,,..., i,, fj... Jnr) a permutation of (1,..., n), is equivalent to 
utor Yim Ay S XQ + +++ +x), OF 


(73) Yi 7 Xig,...5i, € Ao 


Note that the number of such averages with r 7 1 (i.e. omitting the empty 
set of subscripts) is equal to 


È (24) - ("4") -14m 


K=1 


(Problem 57). Thus, H: A = A, is accepted against A > A, at level a = 
1 — //(M + 1) if and only if at least K of the M differences (73) are less 
than Ao, and hence if and only if 8x, < Ao, where 84) < + < ôm) 
denote the ordered set of differences (73). This establishes ôx) as a lower 
confidence bound for A with confidence coefficient y = 1 — a. 

As in the paired comparisons case, it is seen that the intervals (72) each 
have probability 1/(M + 1) of containing A. Thus, two-sided confidence 
intervals and standard confidence points can be derived as before. For the 
generalization to stratified samples, see Problem 58. 

Algorithms for computing the order statistics 95. -++ (my in the paired- 
comparison and two-sample cases are discussed by Tritchler (1984). If M is 
too large for the computations to be practicable, reduced analyses based on 
either a fixed or random subset of the set of all M + 1 permutations are 
discussed, for example, by Gabriel and Hall (1983) and Vadiveloo (1983). 
[See also Problem 60(i).] Different such methods are compared by Forsythe 
and Hartigan (1970). For some generalizations, and relations to other 
subsampling plans, see Efron (1982, Chapter 9). 


15. TESTING FOR INDEPENDENCE IN A BIVARIATE 
NORMAL DISTRIBUTION 


So far, the methods of the present chapter have been illustrated mainly by 
the two-sample problem. As a further example, we shall now apply two of 
the formulations that have been discussed, the normal model of Section 3 


and the nonparametric one of Section 10, to the hypothesis of independence 
in a bivariate distribution. 


| 
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The probability density of a sample (Xj, Y,),...,(X,, Yp) from a bi- 
variate normal distribution is 


(74) 


1 1 1 : 
(rori - 9)" acr cna Eo-o 


= x =4)(y =n) + ZEO z 2 


Here (£, o?) and (n, 7?) are the mean and variance of X and Y respectively, 
and p is the correlation coefficient between X and Y. The hypotheses 
p < Po and p = py for arbitrary pọ cannot be treated by the methods of the 
present chapter, and will be taken up in Chapter 6. For the present, we shall 
consider only the hypothesis p — 0 that X and Y are independent, and the 
corresponding one-sided hypothesis p < 0. 

The family of densities (74) is of the exponential form (1) with 


U-YXxY, TUM, ORLY, =LA =Y, 
and 
ah ə x 
or(1 — ø)’ a5 2e?(1 - P)’ ? 2r(1- p?)’ 

1 £ p De 1 (m & 
a ear am ATP a 
The hypothesis H : p < 0 is equivalent to 6 < 0. Since the sample correla- 
tion coefficient 

E(x - X), - Y) 


Riwear AD 
yxGi - XY L(G - Y) 


is unchanged when the X, and Y; are replaced by (X, — £)/c and (Y, — 1)/7, 
the distribution of R does not depend on £, n, c, or 7, but only on p. For 
6 = 0 it therefore does not depend on #,,..., 94, and hence by Theorem 2, 
R is independent of (T;,..., T4) when 0 = 0. It follows from Theorem 1 
that the UMP unbiased test of H rejects when 


(75) R2Q, 
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or equivalently when 
R 


eG 
\a-R)/(@-2 


The statistic R is linear in U, and its distribution for p = 0 is symmetric 
about 0. The UMP unbiased test of the hypothesis p = 0 against the 
alternative p # 0 therefore rejects when 


(76) 


IRI 
V0 = R?)/(n - 2) 


Since yn — 2R/ V1 — R? has the t-distribution with n — 2 degrees of 
freedom when p = 0 (Problem 64), the constants Kọ and K, in the above 
tests are given by 


(77) > Ky. 


(78) [n0dya and fua) 5. 


Since the distribution of R depends only on the correlation coefficient p, the 
same is true of the power of these tests. 

Paralleling the work of Section 4, let us ask how sensitive the level of the 
test (76) is to the assumption of normality. Suppose that ( X,, Y;),.... 
(X,, Yn) are a sample from some bivariate distribution F with finite second 
moment and correlation coefficient p. In the normal case, the condition 
p = 0 is equivalent to the independence of X and Y. This is not true in 
general, and it then becomes necessary to distinguish between 


H; : X and Y are independent 
and the broader hypothesis that X and Y are uncorrelated, 
H,:p =0. 


Assuming H, to hold, consider the distribution of 


va zn : xr 


E(x-Xy X(-ry 


YnR- 
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Since the distribution of R is independent of = E( X;) and n = E(Y,), 
suppose without loss of generality that £ = ņ = 0. Then the limit distribu- 
tion of Yn (£X,¥,/n) is normal with mean zero and variance 


Var( X,Y,) = E( X2) E(¥?) = 077. 


The term (Vn X)Y tends to zero in probability, since yn X is bounded in 
probability and Y tends to zero in probability. Finally, the denominator 
tends in probability to o7. It follows that Vn R tends in law to the standard 
normal distribution for all F with finite second moments. If a,(F) is the 
rejection probability of the one- or two-sided test (76) or (77) when F is the 
true distribution, it follows that a,(F) tends to the nominal level a as 
n — oo. For studies of how close a,(F) is to a for different F and n, see 
for example Kowalski (1972) and Edgell and Noon (1984). 

Consider now the distribution of yn R under H;. The limit argument is 
the same as under H, with the only difference that Var(X,Y;) need no 
longer be equal to Var X, - Var Y, = o?r?. The limit distribution of Yn R is 
therefore normal with mean zero and variance Var( X;Y;)/[Var X; - Var Y;], 
which can take on any value between 0 and oo (Problem 79). Even 
asymptotically, the size of the tests (76) and (77) is thus completely 
uncontrolled under H;. [It can of course be brought under control by 
appropriate Studentization; see Problem 72 and the papers by Hsu (1949), 
Steiger and Hakstian (1982, 1983), and Beran and Srivastava (1985).] 

Let us now return to H}. Instead of relying on the robustness of R, one 
can obtain an exact level-a unbiased test of independence for a nonpara- 
metric model, in analogy to the permutation test of Section 10. For any 
bivariate distribution of (X, Y), let Y, denote a random variable whose 
distribution is the conditional distribution of Y given x. We shall say that 
there is positive regression dependence between X and Y if for any x < x 
the variable Y,, is stochastically larger than Y,. Generally speaking, larger 
values of Y will then correspond to larger values of X; this is the intuitive 
meaning of positive dependence. An example is furnished by any normal 
bivariate distribution with p > 0. (See Problem 68.) Regression dependence 
is a stronger requirement than positive quadrant dependence, which was 
defined in Chapter 4, Problem 19. However, both reflect the intuitive 
meaning that large (small) values of Y will tend to correspond to large 
(small) values of X. * 1 

As alternatives to H, consider positive regression dependence in a general 
bivariate distribution possessing a probability density with respect to Le- 
besgue measure. To see that unbiasedness implies similarity, let F, F, be 
any two univariate distributions with densities fj, f; and consider the 
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one-parameter family of distribution functions 
(79) F(x) F(y) {1 m A[1 = F,(x)][1 = F,(y)]}, 0xAxI. 


This is positively regression dependent (Problem 69), and by letting A > 0 
one sees that unbjasedness of $ against these distributions implies that the 
rejection probability is a when X and Y are independent, and hence that 


[G3 oi he INA) Gon) On) de dy = a 


for all probability densities f, and f;. By Theorem 6 this in turn implies 
1 
Gaye ete jos) = e. 


Here the summation extends over the (1!)? points of the set S(x, y), which 
is obtained from a fixed point (x, y) with x = (xy,..., Xn) y = (yp... Yn) 
by permuting the x-coordinates and the y-coordinates, each among them- 
selves in all possible ways. 

Among all tests satisfying this condition, the most powerful one against 
the normal alternatives (74) with p > 0 rejects for the k’ largest values of 
(74) in each set S(x, y), where k’/(n!)? = a. Since Yx2, Ly?, Ex,, Ly, are 
all constant on S(x, y), the test equivalently rejects for the k’ largest values 
of Xx, y, in each S(x, y). 

Of the (n!)? values that the statistic Y.X;Y, takes on over S(x, y), only n! 
are distinct, since the statistic remains unchanged if the X's and Y's are 
subjected to the same permutation. A simpler form of the test is therefore 
obtained, for example by rejecting H for the k largest values of Lx iy, of 
each set S(x, y), where xq) < +: < Xen) and k/n! = a. The test can be 
shown to be unbiased against all alternatives with positive regression 
dependence. (See Problem 48 of Chapter 6.) 

In order to obtain a comparison of the permutation test with the 
standard normal test based on the sample correlation coefficient R, let 
T( X,Y) denote the set of ordered X’s and ACE 


HOSE sse keys Yat Ye 
The rejection region of the permutation test can then be written as 


Y XY,» C|T(X, Y)], 
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or equivalently as 
R> K[r(X, Y)]. 


It again turns out* that the difference between K[T(X, Y)] and the 
cutoff point C, of the corresponding normal test (75) tends to zero, and that 
the two tests become equivalent in the limit as n tends to infinity. Sufficient 
conditions for this are that 02,02 » 0 and E(|X|)), E(JY|?) < oo. For 
large n, the standard normal test (75) therefore serves as an approximation 
for the permutation test, which is impractical except for small sample sizes. 


16. PROBLEMS 
Section 2 
l. Let X,,..., X, bea sample from N(£, o°). The power of Student's t-test is an 


increasing function of £/c in the one-sided case H: £ «0, K: > 0, and of 
|£|/c in the two-sided case H: £ = 0, K: $ # 0. 


[If 
1 ERI 
Se EQ) , 


the power in the two-sided case is given by 
cs vng yn(X-§) CS vng 
Poy er ean ed eram 

o 


o o 


and the result follows from the fact that it holds conditionally for each fixed 
value of S/o.] 


2. In the situation of the previous problem there exists no test for testing 
H: £ = 0 at level a, which for all o has power 2 £ > a against the alterna- 
tives ($, o) with £ = $; > 0. 

[Let B(£,, o) be the power of any level a test of H, and let (o) denote the 

power of the most powerful test for testing £ = 0 against £ = $; when o is 
known. Then inf,(£, 9) < inf,B(c) = a.] 

3. (i) Let Z and V be independently distributed as N(6,1) and ? with f 

degrees of freedom respectively. Then the ratio Z + yV/f has the 

noncentral t-distribution with f degrees of freedom and noncentrality 


*For a proof see Fraser (1957). 
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parameter 6, the probability density of which is* 


(8) — pa(t) = Un 


1 oo 
arora” 


xalo) F jJ] dy 


or equivalently 


1 1 f$ 
Ps(t) = | 


20-OP (4p) yaf e|- 2fter 


1 2 
fe Na ince 7 1 8t 
--|v0- —— dv. 

(7 innen es a ar 
Another form is obtained by making the substitution w = tyy / yf in 
(80). T 
Gi) If X,..., X, are independently distributed as N(£,o?), then Vn X 
+ VE(X, — X)'/(n — 1) has the noncentral #-distribution with n — 1 

degrees of freedom and noncentrality parameter ô = yn £/o. 


[(): The first expression is obtained from the joint density of Z and V by 
transforming to t = z + yv/f and v.) 


4. Let X,,..., X, bea sample from N(£, o?). Denote the power of the one-sided 
t-test of H: £ < 0 against the alternative £/o by B(£/a), and by B*(£/a) the 
power of the test appropriate when c is known. Determine B(é/o) for 
n = 5,10,15, a — .05, £/o = 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, and in each case 
compare it with 8*(£/a). Do the same for the two-sided case. 


Let Z,,...,Z, be independently normally distributed with common variance 
o? and means E(Z,) = {,(i = 1,..., s), E(Z,) 2 0 (i 5 s  1,..., n). There 
exist UMP unbiased tests for testing { < t? and § =§? given by the 
rejection regions 


Zot Zg 
ee 1 A |Z, - 8I 


—RR———— >C. 
Y Zh/(n-s) y Y Z/s) 
isl i=s+1 


When f, = $?, the test statistic has the -distribution with n — s degrees of 
freedom. 
*A systematic account of this distrib 


ution can be found in Johnson and Kotz (1970, Vol. 2, 
Chapter 31) and in Owen (1985). 
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Let X,,..., X, be independently normally distributed with common variance 
c? and means £,,...,£,, and let Z; = Y524;X, be an orthogonal transfor- 


mation (that is, £7_,4;;4;, = 1 or Oas j = k or j + k). The Z’s are normally 
distributed with common variance o? and means §; = Xa;/£;. 

[The density of the Z's is obtained from that of the X 's by substituting 
x, = Eb,;z;, where (b;;) is the inverse of the matrix (a,,), and multiplying by 
the Jacobian, which is 1.] 

If X,,..., X, is a sample from N(£, a°), the UMP unbiased tests of £ € 0 and 
£ — 0 can be obtained from Problems 5 and 6 by making an orthogonal 
transformation to variables Z,,...,Z, such that Z, = Yn X. 

[Then 


YyZ-YXz-z-YX--XY(-X] 


i-2 i-1 i=l i=l 


Let X,, X;,... be a sequence of independent variables distributed as N(£, 07), 
and let Y, =["X%41—(%i t: +X,)\/ yn(n-* 1). Then the variables 
Y,, Y,,... are independently distributed as N(0, o?) 


Section 3 


Let X,,..., X, and Y,,...,Y, be independent samples from N(£, o?) and 
N (9, 72) respectively. Determine the sample size necessary to obtain power 
> B against the alternatives 7/0 > A when a = .05, B = .9, ^ = 1.5, 2, 3, and 
the hypothesis being tested is H : t/o <1. 


If m = n, the acceptance region (23) can be written as 


$$ as) 1-C 
ee d se 
mare S2" er G 


where 52 = E(X, — X)", S} = E(Y, — Y)? and where C is determined by 


fe. a 
[800 w= 3: 
0 


Let X,,..., X, and Y,,..., Y, be samples from N(£, o?) and N(n, o°). The 
UMP unbiased test for testing n — § = 0 can be obtained through Problems 5 
and 6 by making an orthogonal transformation from (Xis. ., X,» Y Yn) 
to (Z,,..., Zyn) Such that Z, = (Y -Xy 07m) + 171), Z = (EX + 
EY dm * n. 

Exponential densities. Let X,-- 
exponential density a^! e^" 7^ for x > b. 


., X, be a sample from a distribution with 
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(i) For testing a = 1 there exists a UMP unbiased test given by the accep- 
tance region 


€, x2Y [x - min(x,,...,x,)] « G, 


where the test statistic has a x?-distribution with 2n — 2 degrees of 
freedom when a = 1, and C,, C, are determined by 


[58,300 & = [58,0 y 21 - a. 
€ € 


(i) For testing b = 0 there exists a UMP unbiased test given by the accep- 
tance region 


When b = 0, the test statistic has probability density 


n-l 
(19 u)"' 


p(u) 


uz 0. 


[These distributions for varying b do not constitute an exponential family, and 
Theorem 3 of Chapter 4 is therefore not directly applicable. 

(i): One can restrict attention to the ordered variables Xa € € Xo 
Since these are sufficient for a and b, and transform to new variables 
Z, = nXq), Z -(n-i^ DiX — X 4-1] for i = 2,..., n, as in Problem 14 
of Chapter 2. When a = 1, Z, is a complete sufficient statistic for b, and the 
test is therefore obtained by considering the conditional problem given z,. 
Since 17. , Z, is independent of Z,, the conditional UMP unbiased test has the 
acceptance region C, < E_Z; < C, for each Z,, and the result follows. 

(ii): When b = 0,17 ,Z, isa complete sufficient statistic for a, and the test is 
therefore obtained by considering the conditional problem given Y7..,z,. The 


i=] 


remainder of the argument uses the fact that Z, /7 Z, is independent of 


i=1 
j.,Z, when b = 0, and otherwise is similar to that used to prove Theorem 1.] 


Extend the results of the preceding problem to the case, considered in Problem 


10, Chapter 3, that observation is continued only until X,),..., Xg) have been 
observed, 


Section 4 
Corollary 2 remains valid if c, is replaced by a sequence of random variables 
C, tending to c in probability. 


G) Let X,,..., X, be a sample from N(£, 07). The power of the one-sided 


one-sample t-test against a sequence of alternatives (£,, 0) for which 
Vn£,/a — 8 tends to (8 — u,). 
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16. 
17. 


18. 


19. 


20. 
21. 
22. 
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(i) The result of (i) remains valid if X,,..., X, are a sample from any 
distribution with mean £ and finite variance o°. 


Generalize Problem 15(i) and (ii) to the two-sample t-test. 


(i) Given p, find the smallest and largest value of (31) as o?/7? varies from 
0 to oo. 

(ii) For nominal level a = .05 and p = 1, 2, .3,.4, determine the smallest and 
the largest asymptotic level of the t-test as 02/7? varies from 0 to oo. 


Section 5. 
The Chebyshev inequality. For any random variable Y and constants a > 0 
and c, 


E(Y - c)! > a?P(lY — e| > a). 


If Y, is a sequence of random variables and c a constant such that E(Y, — c)? 
> 0, then for any a > 0, 


P(IY, - d a) > 0, 


that is, Y, tends to c in probability. 
Verify the formula for Var(X) in Model A. 


In Model A, suppose that the number of observations in group i is nj. If 
n, € M and s > oo, show that the assumptions of Lemma 1 are satisfied and 


determine y. 


Show that the conditions of Lemma 1 are satisfied and y has the stated value: 
(i) in Model B; (ii) in Model C. 


Determine the maximum asymptotic level of the one-sided (t-test when a = .05 
and m — 2,4,6: (i) in Model A; (ii) in Model B. 


Let X, = ¢ + U, and suppose that the joint density of the U's is spherically 
symmetric, that is, a function of EU? only, 


f...) = (Xi). 


Then the null distribution of the one-sample t-statistic is independent of q and 
hence the same as in the normal case, namely Student's ¢ with n — 1 degrees 


of freedom. 
[Write 7 as 
Yn x/ (Ex: 
E- Eyin- Deg 
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and use the fact that when £ = 0, the density of X,,..., X, is constant over 
the spheres Xx? = c and hence the conditional distribution of the variables 
X/ (Ex given DX? = c is uniform over the conditioning sphere and hence 
independent of q.] 

Note. This model represents one departure from the normal-theory assump- 
tion, which does not affect the level of the test. The effect of a much weaker 
symmetry condition more likely to arise in practice is investigated by Efron 
(1969). 


. Section 6 


On the basis of a sample X = (Xj,..., X,) of fixed size from N(£, o?) there 
do not exist confidence intervals for £ with positive confidence coefficient and 
of bounded length. 

[Consider any family of confidence intervals 8( X) + L/2 of constant length 
L. Let &,..., £y be such that |£, — | > L whenever i + j. Then the sets 
S, = (x: |8(x) - &| < L/2) (i — 1,...,2N) are mutually exclusive. Also, 
there exists oj > 0 such that 


1 
|, QOe S) - Ff KES} s zx for o> €, 


as is seen by transforming to new variables Y, = (X, — §,)/o and applying 
Lemmas 2 and 4 of the Appendix. Since min, P ,(X€ $) < 1/2N, it 
follows for ø > a, that min, P. ,(X € S;} < 1/N, and hence that 


E 1 
inf P, ,A|8( X) — = = 
B ‘| (X) - & < 5} X 
The confidence coefficient associated with the intervals 6(X) + L/2 is there- 
fore zero, and the same must be true a fortiori of any set of confidence 
intervals of length « L.] 


Stein's two-stage procedure. 


G) If mS? /o? has a x? = distribution with m degrees of freedom, and if the 
conditional distribution of Y given S — s is N(0, o?/S?), then Y has 
Student's r-distribution with m degrees of freedom. 

(ii) Let Xs Xs be independently distributed as N(¢, 02). Let X = 
Ej2,X/no, S* = Y7e,(X, — X)*/(mg — 1), and let gum rrr ma, T 


a, 
4, d4,41 = ``: =a,=b, and n> ny be measurable functions of S. 
Then 
n 
£ a,( X, — £) 
y- iz 


7 
you a; 
i=l 


has Student’s distribution with ny — 1 degrees of freedom. 
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Consider a two-stage sampling scheme TI}, in which S? is computed from 
an initial sample of size no, and then n — n, additional observations are 
taken. The size of the second sample is such that 


s? 
nos [7] * i 


where c is any given constant and where [y] denotes the largest integer 
<y. There then exist numbers 4,,...,a, such that a, = ::: = 
aus Ong totom dg, Dimi; = 1, Lra? = c/S?. It follows from (ii) 
that E^. ,a,(X, — £)/ Vc has Student's i-distribution with n, — 1 degrees 
of freedom. 

The following sampling scheme II,, which does not require that the 
second sample contain at least one observation, is slightly more efficient 
than II, for the applications to be made in Problems 27 and 28. Let no, 
S?, and c be defined as before; let 


atibus. 


a, =1/n (i =1,..., n), and X = Y/.,a, X,. Then Vn (X —£)/S has again 
the ¢-distribution with ny — 1 degrees of freedom. 


[(ii): Given S = s, the quantities a, b, and n are constants, £72 ,a,(X, — §) = 
noa(X, — £) is distributed as N(0,noa7o”), and the numerator of Y is 


therefore normally distributed with zero mean and variance o?Y" 1a. The 
result now follows from (i).] 


27. Confidence intervals of fixed length for a normal mean. 


G) 


Gi) 


In the two-stage procedure II, defined in part (iii) of the preceding 
problem, let the number c be determined for any given L> 0 and 


0<y<1by 


[PF ugar) nr. 
-L/f 


where t... denotes the density of the t-distribution with ny — 1 degrees 
of freedom, Then the intervals D?_,a;X, + L/2 are confidence intervals 
for £ of length L and with confidence coefficient y. 

Let c be defined as in (i), and let the sampling procedure be II, as 
defined in part (iv) of Problem 26. The intervals X X L/2 are then 
confidence intervals of length L for £ with confidence coefficient > y, 
while the expected number of observations required is slightly lower than 
under II,. 
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[@: The probability that the intervals cover £ equals 


n 


p "ECHO 6, 


BL 3500 FOREESQQET 3 SA 


LP 


(ii): The probability that the intervals cover £ equals 


e on 


Two-stage t-tests with power independent of o. 


(i) For the procedure IT, with any given c, let C be defined by 
' 0 
fta) & = a. 
c 


Then the rejection region (Z7. a; X, — £))/ Vc > C defines a level-a test 
of H: $ < £y with strictly increasing power function B,(£) depending 
only on £. 


(ii) Given any alternative £, and any a < B « 1, the number c can be 
chosen so that &,(£,) = f. 


(iii) The test with rejection region Vn (X —£,)/S > C based on II, and the 
same c as in (i) is a level-a test of H which is uniformly more powerful 
than the test given in (i). 


(v) Extend parts (i)-(iii) to the problem of testing $ = £, against £ + £o. 
[(i) and (ii): The power of the test is 


SOLE COS 


t, 
(E-£0)/ E 


(ii): This follows from the inequality Vn |£ — £y/S  £ — £|/ Ve] 


Let S(x) be a family of confidence sets for a real-valued parameter ĝ, and let 
pI S(x)] denote its Lebesgue measure. Then for every fixed distribution Q of X 
(and hence in particular for Q — Pj, where @ is the true value of 0) 


Eo(n[S(X)]) = Pk Q(6 e s(X)) dð 


provided the necessary measurability conditions hold. 
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30. 


SP 


32. 


33. 


34. 
35. 


36. 


[Write the expectation on the left side as a double integral, apply Fubini's 
theorem, and note that the integral on the right side is unchanged if the point 
0 = 6, is added to the region of integration.] 


Use the preceding problem to show that uniformly most accurate confidence 
sets also uniformly minimize the expected Lebesgue measure (length in the 
case of intervals) of the confidence sets.* 


Section 7 


Let X,,..., X, be distributed as in Problem 12. Then the most accurate 
unbiased confidence intervals for the scale parameter a are 


—X[x -min(x,...,x,)] sas ZE = min(x,,...,x,)]. 


Most accurate unbiased confidence intervals exist in the following situations: 

(i) If X,Y are independent with binomial distributions b(p,,m) and 
b( pz, n), for the parameter p,q2/p29- 

(i) Ina 2 X 2 table, for the parameter A of Chapter 4, Section 6. 

Section 8 

(i) Under the assumptions made at the beginning of Section 8, the UMP 
unbiased test of H : p = pọ is given by (45). 

(ii) Let (p, p) be the associated most accurate unbiased confidence intervals 
for p = ay + bô, where p = p(a, b), p = p(a, b). Then if f, and f, are 
increasing functions, the expected value of f,(|p — pl) + f;(|p — p) is an 
increasing function of a?/n + b°. 


[): Make any orthogonal transformation from y;,..., y, to new variables 
2,..., Z, Such that z; = E,[bv; + (a/n)]y/ V (a?/n) + b°, z, = X, (av, - 


b) y,/ Va? + nb? , and apply Problems 5 and 6. y d 
(ii): If a2/n + b? < a2/n + bj, the random variable |p(a;, b;) — p| is sto- 


chastically larger than [p(a;, b,) — p|, and analogously for p.] 
Section 9 
Verify the posterior distribution of 8 given x in Example 12. 


If X,,..., X, are independent N(6,1) and @ has the improper prior (0) = 1, 
determine the posterior distribution of 8 given the X’s. 


Verify the posterior distribution of p given x in Example 13. 


*For the corresponding result concerning one-sided confidence bounds, see Madansky 


(1962). 
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In Example 14, verify the marginal posterior distribution of £ given x. 
In Example 15, show that 


G) the posterior density 7(o|x) is of type (c) of Example 13; 
(i) for sufficiently large r, the posterior density of o” given x is no longer of 
type (c). 


If X is normal N(0,1) and @ has a Cauchy density b/{m[b? + (0 — n. 
determine the possible shapes of the HPD regions for varying p and b. 


Let 0 = (0;,..-,9,) with 6, real-valued, X have density p(x), and © a prior 
density 7(8). Then the 100y% HPD region is the 100y% credible region R 
that has minimum volume. 

[Apply the Neyman-Pearson fundamental lemma to the problem of minimiz- 
ing the volume of R.] 


Let X,,..., X, and Y,,..., Y, be independently distributed as N(£, o°) and 
N(n, o?) respectively, and let ($, 7, o) have the joint improper prior density 
given by 


1 
n(£,n,0) d£ dn do = dé dn - z do forall -œ <¢,ņn < %, O0<o. 


Under these assumptions, extend the results of Examples 14 and 15 to 
inferences concerning (i) ņ — £ and (ii) o. 


Let X,,..., Xm and Y,,..., Y, be independently distributed as N(é, c?) and 
N(n, 72), respectively and let (£, n, o, 7) have the joint improper prior density 
n (£, n, a, 7) d£ dn do dr = d£ dn (1/0) do (1/7) dr. Extend the result of Ex- 
ample 15 to inferences concerning 17/0”. 

Note. The posterior distribution of 5 — £ in this case is the so-called 
Behrens-Fisher distribution. The credible regions for 1 — $ obtained from this 
distribution do not correspond to confidence intervals with fixed coverage 
probability, and the associated tests of H: n = £ thus do not have fixed size 
(which instead depends on r/o). From numerical evidence [see Robinson 
(1976) for a summary of his and earlier results] it appears that the confidence 


intervals are conservative, that is, the actual coverage probability always 
exceeds the nominal one. 


Let T;,...,7,-; have the multinomial distribution (34) of Chapter 2, and 
suppose that (Py. -, p, 1) has the Dirichlet prior density D(a;,..., 45) with 
density proportional to p^-!... p=}, where p, -1— (pi: +Ps-1)- 


Determine the posterior distribution of ( p, ... , p, 1) given the T's. 


Section 10 
Prove Theorem 6 for arbitrary values of c. 


5.16] PROBLEMS 263 
Section 11 
45. If c=1, m=n=4, a= .1 and the ordered coordinates 5... 2) ofa 


point z are 1.97, 2.19, 2.61, 2.79, 2.88, 3.02, 3.28, 3.41, determine the points of 
S(z) belonging to the rejection region (54). 


46. Confidence intervals for a shift. 


() Let Xiro X5 Yo Ys be independently distributed according to 
continuous distributions F(x) and G(y) = F(y — A) respectively. 
Without any further assumptions concerning F, confidence intervals for A 
can be obtained from permutation tests of the hypotheses H(Ag): A = Ao. 
Specifically, consider the gx E (25. Zman) = (Qo Xm 7 
A,..-5 Ya — 4) and the (” z permutations i < `° < imi Im+i < 
+++ < ime Of the integers 1,..., m + n. Suppose that the hypothesis 
H(A) is accepted for the k of these permutations which lead to the 
smallest values of 


m+n m 


X a/n- La/m 


j=m+1 j=1 


where 
k-(1- o(" 5, D 


Then the totality of values A for which H(A) is accepted constitute an 
interval, and these intervals are confidence intervals for A at confidence 
level 1 — a. 

(ii) Let Z,,...,Zy be independently distributed, symmetric about 0, with 
distribution F(z — 6), where F(z) is continuous and symmetric about 0. 
Without any further assumptions about F, confidence intervals for 6 can 
be obtained by considering the 2" points Zj,.--, Zi, where Z/ = +(Z; 
— 65), and accepting H(Q): 9 = 9% for the k of these points which lead 
to the smallest values of £|Z/|, where k = (1 — «) 2". 


[@: A point is in the acceptance region for H(A) if 


pacem. 


— | =|p- 3-4 
n m 


is exceeded by at least Uer 2j — k of the quantities |J’ — x’ — yA|, where 
(EA Phe zin Pisces: as a permutation of (x; -» Xy» po Jn)» the quan- 
tity y is determined by this permutation, and |y] < 1. The desired result now 
follows from the following facts (for an alternative proof, see Section 14): (a) 
The set of A's for which (y — X — My (yx yA)? is, with probability 


one, an interval containing j — X. (b) The set of A's for which (y — X — A)? 


264 


47. 


48. 


49. 


50. 


UNBIASEDNESS: APPLICATIONS; CONFIDENCE INTERVALS [5.16 


is exceeded by a particular set of at least im z) — k of the quantities 
(y' — X' — yA)? is the intersection of the corresponding intervals (a) and 
hence is an interval containing y — X. (c) The set of A's of interest is the union 
of the intervals (b) and, since they have a nonempty intersection, also an 
interval.] 


Section 12 


In the matched-pairs experiment for testing the effect of a treatment, suppose 
that only the differences Z, — Y, — X, are observable. The Z's are assumed to 
be a sample from an unknown continuous distribution, which under the 
hypothesis of no treatment effect is symmetric with respect to the origin. Under 
the alternatives it is symmetric with respect to a point { > 0. Determine the 
test which among all unbiased tests maximizes the power against the alterna- 
tives that the Z's are a sample from N(£, o?) with ¢ > 0. 

[Under the hypothesis, the set of statistics (Z’_,Z?,...,L7-1Z/") is sufficient; 
that it is complete is shown as the corresponding result in Theorem 6. The 
remainder of the argument follows the lines of Section 11.] 


(i) If X,,..., X,; Yy,..., Y, are independent normal variables with common 
variance o? and means E(X,) = £j, E(Y,) = £, + A, the UMP unbiased 
test of A = 0 against A > 0 is given by (59). 

(ii) Determine the most accurate unbiased confidence intervals for A. 


[@: The structure of the problem becomes clear if one makes the orthogonal 
transformation X/ = (Y, — X,)/V2, Y/ = (X, + Y)/V2.] 


Comparison of two designs. Under the assumptions made at the beginning of 
Section 12, one has the following comparison of the methods of complete 
randomization and matched pairs. The unit effects and experimental effects U; 
and V, are independently normally distributed with variances oj, o? and 
means E(U,) = and E(V,) =€ or ņ as V, corresponds to a control or 
treatment. With complete randomization, the observations are X, = U, + V; 
(i 1,..., n) for the controls and Y, = U,,, + V,,; (i L,..., n) for the 
treated cases, with E(X,) = p + £, E(Y,) = p + 7. For the matched pairs, if 
the matching is assumed to be perfect, the X's are as before, but Y, = U, + 
V, ,;. UMP unbiased tests are given by (27) for complete randomization and 
by (59) for matched pairs. The distribution of the test statistic under an 
alternative A — 5 —£ is the noncentral f-distribution with noncentrality 
parameter yn A/ V2(a? +0?) and 2n — 2 degrees of freedom in the first 
case, and with noncentrality parameter /n 4/26 and n— 1 degrees of 
freedom in the second. Thus the method of matched pairs has the disadvantage 
of a smaller number of degrees of freedom and the advantage of a larger 
noncentrality parameter. For a = .05 and A = 4, compare the power of the 
two methods as a function of n when o; = 1, o = 2 and when o, = 2, o = 1. 


Continuation. An alternative comparison of the two designs is obtained by 
considering the expected length of the most accurate unbiased confidence 
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intervals for A=1—€ in each case. Carry this out for varying m and 
confidence coefficient 1 — a = .95 when o — 1, o —2 and when o, = 2, 
o=1, 


Section 13 


51. Suppose that a critical function $o satisfies (65) but not (67), and let a < i. 
Then the following construction provides a measurable critical function 4 
satisfying (67) and such that $)(z) < $(z) for all z. Inductively, sequences of 
functions $;, $,,... and jo, J4.... are defined through the relations 


$,(z) 
E Caecina esseri aT Rete 
rese) M! Ne! 


and 


$m—1(2) + [a 7 Ym-1(2)] 
"TOP if both $, .,(z) and V, ,(z) are <a, 
$,—,(z) otherwise. 


The function $(z) = lim $, (z) then satisfies the required conditions. 

[The functions ¢,, are nondecreasing and between 0 and 1. It is further seen by 
induction that 0 < a — ¥,,(z) < (1 — y)"[a — ¥o(z)], where y= 1/ 
N,!... NL] 

52. Consider the problem of testing H : ņ = £ in the family of densities (62) when 
it is given that o > c > 0 and that the point (Sirs-+ ++ Sew.) of (63) lies in a 
bounded region R containing a rectangle, where c and R are known. Then 
Theorem 7 is no longer applicable. However, unbiasedness of a test $ of H 
implies (67), and therefore reduces the problem to the class of permutation 


tests. 
[Unbiasedness implies /6(z) p,, (2) dz = a and hence 


1 1 
am JWE) p aO) de = f) Her 25 EE CH - t) 


for all ø > c and ¢ in R. The result follows from completeness of this last 
family.] 

53. To generalize Theorem 7 to other designs, let Z = (Z,,..., Zw) and let 
G= (g,...,g,) be a group of permutations of N coordinates or more 
generally a group of orthogonal transformations of N-space. If 


1 1 1 5 
(81) Po.¢(2) = = » a Ea ) 


k=1 
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where |z|? = Xz2, then fó(z) pa, p(z) dz < a for all o > 0 and all f implies 
1 
(82) —- Y e(z)sa ae, 
T zes(z) 


where S(z) is the set of points in N-space obtained from z by applying to it 
all the transformations g,, k = 1,..., r. 


Generalization of Corollary 3. Let H be the class of densities (81) with o > 0 
and — œ < f, < œ (i = 1,..., N). A complete family of tests of H at level of 
significance a is the class of permutation tests satisfying 


1 
(83) = Y e(z)-a ae. 
 ses(z) 
Section 14 


If c=1, m=n=3, and if the ordered x's and y's are respéctively 
1.97,2.19,2.61 and 3.02,3.28, 3.41, determine the points 3,,..., 5:19) defined 
as the ordered values of (73). 


If c — 4, m, 7 n; — 1, and the pairs (x,, y;) are (1.56,2.01), (1.87,2.22), 
(2.17,2.73), and (2.31,2.60), determine the points §,),..., 8,5, which define 
the intervals (72). 


If m, n are positive integers with m « n, then 


X (Rk) -("27) a: 


K=1 


(i) Generalize the randomization models of Section 14 for paired compari- 
sons (m = ::- =n, = 2) and the case of two groups (c = 1) to an 
arbitrary number c of groups of sizes m,,..., ne- 

(ii) Generalize the confidence intervals (72) and (73) to the randomization 
model of part (i). 


Let Z,,...,Z, be iid. according to a continuous distribution symmetric 
about 0, and let Ta) < --- < Ty) be the ordered set of M = 2" — 1 
subsamples (Z, + +-- +Z,)/r, r > 1. If To = — 00, Trays) = oo, then 


1 
Pol Tn <9 < Tuan] = 5G forall i=0,1,...,M. 
(Hartigan (1969).] 


(i) Given n pairs Cn, iss, Yn) let G be the group of 2" permuta- 
tions of the 2" variables which interchange x, and y, in all, some, or none 
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of the n pairs. Let G) be any subgroup of G, and let e be the number of 
elements in Gp. Any element g € Gy (except the identity) is characterized 
by the numbers i,,...,i, (r > 1) of the pairs in which x, and y, have 
been switched. Let d; = y, — x;, and let à, < --- < &,., denote the 
ordered values (d; + +- +d; )/r corresponding to Go. Then (72) con- 
tinues to hold with e — 1 in place of M. 


(ii) State the generalization of Problem 59 to the situation of part (i). 


[Hartigan (1969).] 


The preceding problem establishes a 1:1 correspondence between e — 1 
permutations T of Gy which are not the identity and e — 1 nonempty subsets 
{i,,...5i,} of the set (1,..., n). If the permutations T and T’ correspond 
respectively to the subsets R = {i;,...,i,} and R' = (A), then the 
group product T’T corresponds to the subset (RMS) U (RO S)-(RUS) 
— (R N S). [Hartigan (1969).] 

Determine for each of the following classes of subsets of (1,..., n} whether 


(together with the empty subset) it forms a group under the group operation of 
the preceding problem: All subsets {i,,-.., i,} with 


© r22; 
(i) r = even; 
(iii) r divisible by 3. 
(v) Give two other examples of subgroups Go of G. 
Note. A class of such subgroups is discussed by Forsythe and Hartigan 
(1970). 


Generalize Problems 60(i) and 61 to the case of two groups of sizes m and n 
(c=1). 
Section 15 
G) If the joint distribution of X and Y is the bivariate normal distribution 
(70), then the conditional distribution of Y given x is the normal 
distribution with variance 7^(1 — p^) and mean y + (p7/o)(x — £). 
(ii) Let (X, Y,),...,(X,, Yp) be a sample from a bivariate normal distribu- 
tion, let R be the sample correlation coefficient, and suppose that p — 0. 
Then the conditional distribution of yn — 2 R/ V1 — R? given Xy... Xn 


is Student's distribution with m — 2 degrees of freedom provided 
L(x, — X) > 0. This is therefore also the unconditional distribution of 


this statistic. 
Gii) The probability density of R itself is then 


1 r[iG- 2] 


(84) p(r)= CED =p). 
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(i): If v = (x, — x)/ E(x; — x)’ so that Xv, = 0, Ev; = 1, the statistic 


can be written as 
Lay, 
(fx - nF - ( (Eux) Yype-»- 


Since its distribution depends only on p one can assume 9 = 0, 7 = 1. The 
desired result follows from Problem 6 by making an orthogonal transformation 
from (Y;,..-, Y;) to (Zis: Z,) such that Z, = Vn Y, Z, = EuY,] 


(i) Let(X,, Y,),..., (Xn, Y,) be a sample from the bivariate normal distribu- 
tion (70), aad: let S2 2 E(X, - Xy, S2 =X(Y, - Y), $; -X(X,- 
XXY, — Y). There exists a UMP unbiased test for testing the hypothesis 
t/o = A. Its acceptance region is 


Ws; — Sil 


Jepgo iem Nem AN 
(4st + Si) - «si 
and the probability density of the test statistic is given by (84) when the 
hypothesis is true. 

(i) Under the assumption 7 = o, there exists a UMP unbiased test for testing 
n = £, with acceptance region |F — X|/ yS? + S? — 25,; < C. On multi- 
plication by a suitable constant the test statistic has Student's t-distribu- 
tion with n — 1 degrees of freedom when 7 = £. (Without the assumption 


7 = øo, this hypothesis is a special case of the one considered in Chapter 8, 
Example 2.) 


[(i): The transformation U = AX + Y, V = X — (1/A)Y reduces the problem 
to that of testing that the correlation coefficient in a bivariate normal distribu- 
tion is zero. 

(ii): Transform to new variables V, = Y, — X,, U; = Y; + X] 


G) Let(X,, Y),..., (X, n, Yobe sample. from the bivariate normal : distribu- 
tion (74), and ‘Tet S? -Y(X,- Xy, S5 - XX, - XXY,- Y, ST = 
Ly, - Y). 


Then (SẸ, S,2, S2) are independently distributed of (X, m and their joint 
distribution is ise same as that of (7-1 X7, E/Z1X/Y/, Dra ¥/), where 
s Y), i=1,. — 1, are a sample from the distribution (74) with £ = 7 


(i) Let X,...,X, and Y,,..., Y, be two samples from N(0,1). Then the 
joint density of S? = YX2, S =LX,¥,, S = LY? is 


2 jimn-» 


Gal (m 1) 153 7 5h) exp[ -1(s? *si)] 


for s2, < 5252, and zero elsewhere. 
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(ii) The joint density of the statistics (S?, S,2, S2) of part (i) is 


(85) 


Bigs 
e re- 1 E M d 
4sT(n-2(emi-g) - 20 - 6°) 


c? oT T 
for sł, < s7s3, and zero elsewhere. 


[(i): Make an orthogonal transformation from X;,..., X, to X{,..., X; such 
that X; = Yn X, and apply the same orthogonal transformation also to 
Yj,--:5 33: een 


x-4YT Yxx-EQ-30-?». 


i=l i=l 
n-1 n c n-1 n x 
Xanh EY.-»:vo-r. 
i=l i=l i=l i=l 


The pairs of variables (Xj, Y7), ..., (Xp, Yn) are independent, each with a 
bivariate normal distribution with the same variances and correlation as those 
of (X, Y) and with means E(X/) = E(Y/) = 0 for i =1,..., n — 1. 

Gi): Consider first the joint distribution of S; = Ex; Y, and $2 = XY? given 
Xj,..., Xm. Letting Z; = Sj2/ (£x and making an orthogonal transforma- 
tion from Y,,..., Y, tO Z,,---» Zm so that S7 = Y7.1Z], the variables Z, and 
E", Z? = S2 — Z? are independently distributed as N(0,1) and x2, , re- 
spectively. From this the joint conditional density of Sı, = s,Z, and 82i 
obtained by a simple transformation of variables. Since the conditional distri- 
bution depends on the x's only through s2, the joint density of SẸ, S15, S2 is 
found by multiplying the above conditional density by the marginal one of SO 
which is x2,. The proof is completed through use of the identity 


m-T(m-1 
riie - o]rüo) - 2. 


(iii): If (X7, Y^) = (Xf, Y... 5X5, Ya) is a sample from a bivariate normal 
distribution with £ = 4 = 0, then T= OKL LXV), LY”) is sufficient for 
0 = (c, p, 7), and the density of T is obtained from that given in part (ii) for 
&, = (1,0,1) through the identity [Chapter 3, Problem 14 (à 


XY yt. yh 

pi (xy) 

7 T 

Pit) PAD 
"o puo QS») 


The result now follows from part (i) with m — n — 1] 
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67. If (X, Yi), -- -s (X,, Yn) is a sample from a bivariate normal distribution, the 
probability density of the sample correlation coefficient R is* 


(86) P(r) = ayl- pye?a - pyr 


E: 


x È T?[(n 4 k — 1)] 
or alternatively 


(87) nr) = Ža- gy - eye 
xf. n m i 
ELE—EE d 
o (1—prt)" ^ 1- à 
Another form is obtained by making the transformation ¢ = (1 — v)/(1 — prv) 
in the integral on the right-hand side of (87). The integral then becomes 
i ux 2 
(88) Xue ew ruo ~ = [i - 401 + or)] ^ av. 


Expanding the last factor in powers of v, the density becomes 


(89) "UR rc) C E o RECO TE NM 


l4 pr 
xristian- a ) 


2 


where 


(90) F(a;b,c x) EU Iun) WAR) eL C INA 


joo T(a)  I(5) Tet) j! 


is a hypergeometric function. 
[To obtain the first expression make a transformation fr 

om (S2, S2, S12) with 
density (85) to (S, S2, R) and expand the factor exp(ps;5/(1 ze p)or) = 


*The distribution of R is reviewed by Johnson and Kotz i d 
PARE and Kotz (1970, Vol. 2, Section 32) ant 
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exp(prs,s;/(1 — p)ar} into a power series. The resulting series can be 
integrated term by term with respect to s? and s2. The equivalence with the 
second expression is seen by expanding the factor (1 — prt)- "9 under the 
integral in (87) and integrating term by term.] 

68. If X and Y have a bivariate normal distribution with correlation coefficient 
p > 0, they are positively regression-dependent. 
[The conditional distribution of Y given x is normal with mean 7 + pro^ ! (x 
— £) and variance 7^(1 — p°). Through addition to such a variable of the 
positive quantity pra^! (x' — x) itis transformed into one with the conditional 
distribution of Y given x’ > x.] 

69. (i) The functions (79) are bivariate cumulative distributions functions. 
(ii) A pair of random variables with distribution (79) is positively regression- 

dependent. 

70. If X,Y are positively regression dependent, they are positively quadrant 
dependent. 
[Positive regression dependence implies that 


(91) P[Y <y|X<x]=P[¥sy|X<x'] forall x< x' and y, 


and (91) implies positive quadrant dependence.] 


7l. There exist bivariate distributions F of (X,Y) for which p=0 and 
Var( XY )/[Var( X)Var(Y)] takes on any given positive value. 


Additional Problems 
72. Let (X,, Y), i= L,..., n, be iid. according to a bivariate distribution F with 
E(X?), E(P) < oo. 
(i) If R is the sample correlation coefficient, then Yn R is asymptotically 
normal with mean 0 and variance Var( X, Y;)/Var X, Var Y,. 
(i) The variance of part (i) can take on any value between 0 and oo. 
Gii) For testing H,:p = against p > 0, define a denominator D, and 
critical value c, such that the rejection region R/D, > c, has probability 
a, (F) > a for all F satisfying H2. 
73. Shape parameter of a gamma distribution. Let Xy,..., X, bea sample from 
the gamma distribution T'(g, b) defined in Problem 43 of Chapter 3. 


(i) There exist UMP unbiased tests of H:g < go against g> go and of 
H’: g = go against g # go, and their rejection regions are based on 
w - (XX). ; 

(ii) There exist uniformly most accurate confidence intervals for g based 
on W. 
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[Shorack (1972).] 

Notes. 

(1) The null distribution of W is discussed in Bain and Engelhardt (1975), 
Glaser (1976), and Engelhardt and Bain (1978). 

(2) For g — 1, I'(g, b) reduces to an exponential distribution, and (i) be- 
comes the UMP unbiased test for testing that a distribution is exponential 
against the alternative that it is gamma with g > 1 or with g * 1. 

(3) An alternative treatment of this and some of the following problems is 
given by Bar-Lev and Reiser (1982). 


74. Scale parameter of a gamma distribution. Under the assumptions of the 
preceding problem, there exists 


(i) A UMP unbiased test of H: b < bọ against b > by which rejects when 
EX» CMIX). 
(ii) Most accurate unbiased confidence intervals for b. 


[The conditional distribution of E X, given [I X,, which is required for carrying 
out this test, is discussed by Engelhardt and Bain (1977).] 


75. Gamma two-sample problem. Let X,,..., Xm; Yi,..., Y, be independent sam- 
ples from gamma distributions T (g;, 5), '(g2, b;) respectively. 


(i) If gı, g; are known, there exists a UMP unbiased test of H : b; = b, 
against one- and two-sided alternatives, which can be based on a beta 
distribution. 

[Some applications and generalizations are discussed in Lentner and 
Buehler (1963).] 


(ii) If g,, g, are unknown, show that a UMP unbiased test of H continues to 
exist, and describe its general form. 


(iii) If b; = b, = b (unknown), there exists a UMP unbiased test of g; = £i 
against one- and two-sided alternatives; describe its general form. 


{@): If Y, (i = 1,2) are independent T(g;, b), then hj + Y, is T(g; + 2,5) 
and Y,/(Y, + Y;) has a beta distribution.] 


76. Let X,,..., X, bea sample from the Pareto distribution P(c, 7), both parame- 
ters unknown. Obtain UMP unbiased tests for the parameters c and 7. 


[Problem 12, and Problem 44 of Chapter 3.] 


TT. Inverse Gaussian distribution. Let X,,..., X, be a sample from the inverse 
Gaussian distribution I(u, 7), both parameters unknown. 


(i) There exists a UMP unbiased test of p < pọ against p> po, which 
rejects when X > C[E(X, + 1/X,)], and a corresponding UMP unbiased 


*For additional information concerning inference in inverse Gaussian distributions, see 
Folks and Chhikara (1978), 


5.17] REFERENCES 273 


test of u = fo against p * po- 
[The conditional distribution needed to carry out this test is given by 
Chhikara and Folks (1976).] 

(ii) There exist UMP unbiased tests of H:1 = % against both one- and 
two-sided hypotheses based on the statistic V = X(1/X, — 1/X). 

Gii) When r = 1, the distribution of 7V is x7. 


[Tweedie (1957).] 


78. Let X,,..., X, and Y,...,Y, be independent samples from (pu, 0) and 
I(v, 7) respectively. 


G) There exist UMP unbiased tests of 7/7, against one- and two-sided 
alternatives. 

(ii) If +=, there exist UMP unbiased tests of v/j against one- and 
two-sided alternatives. 


[Chhikara (1975).] 


79. Consider a one-sided, one-sample, level-a t-test with rejection region t( X) > c,, 
where X = (X,,..., Xa) and 1(X) is given by (16). Let a, (F) be the rejection 
probability when X,,..., X, are ii.d. according to a distribution F € F, with 
F the class of all distributions with mean zero and finite variance. Then for 
any fixed n, no matter how large, supp e x &,(F) = 1. 

[Let F be a mixture of two normals, F = yNQ, o?) + Q — Y)N(p, o?) with 
y+(1-— y)n = 0. By taking y sufficiently close to 1, one can be virtually 
certain that all n observations are from N(1, o?). By taking o sufficiently 
small, one can make the power of the t-test against the alternative N(1, o?) 
arbitrarily close to 1. The result follows.] 

Note. This is a special case of results of Bahadur and Savage (1956); for 
further discussion, see Loh (1985). 
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CHA PAISEARIGÓ 


Invariance 


1. SYMMETRY AND INVARIANCE 


Many statistical problems exhibit symmetries, which provide natural restric- 
tions to impose on the statistical procedures that are to be employed. 
Suppose, for example, that X,,..., X, are independently distributed with 
probability densities py (x;),..., Po (x,). For testing the hypothesis H : 0, 
= -+- = 6, against the alternative that the @’s are not all equal, the test 
should be symmetric in x,...,x,, since otherwise the acceptance or 
rejection of the hypothesis would depend on the (presumably quite irrele- 
vant) numbering of these variables. 

As another example consider a circular target with center O, on which 
are marked the impacts of a number of shots. Suppose that the points of 
impact are independent observations on a bivariate normal distribution 
centered on O. In testing this distribution for circular symmetry with respect 
to O, it seems reasonable to require that the test itself exhibit such 
symmetry. For if it lacks this feature, a two-dimensional (for example, 
Cartesian) coordinate system is required to describe the test, and acceptance 
or rejection will depend on the choice of this system, which under the 
assumptions made is quite arbitrary and has no bearing on the problem. 

The mathematical expression of symmetry is invariance under a suitable 
group of transformations. In the first of the two examples above the group is 
that of all permutations of the variables Xy-.., X, Since a function of n 
variables is symmetric if and only if it remains invariant under all permuta- 
tions of these variables. In the second example, circular. symmetry 
with respect to the center O is equivalent to invariance under all rotations 
about O. 

In general, let X be distributed according to a probability distribution 
Py, 0 € Q, and let g be a transformation of the sample space Z. All such 
transformations considered in connection with invariance will be assumed 
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to be 1:1 transformations of % onto itself. Denote by gX the random 
variable that takes on the value gx when X = x, and suppose that when the 
distribution of X is Pj, 0 € Q, the distribution of gX is Py, with 6^ also in 
Q. The element 0^ of Q which is associated with @ in this manner will be 
denoted by g6, so that 


(1) P,(gX € A) = Pu(X€ A}. 


Here the subscript 0 on the left member indicates the distribution of X, not 
that of gX. Equation (1) can also be written as Py(g !4) = Pg( A) and 
hence as 


(2) Pa(g4) = P&(A). 


The parameter set 2 remains invariant under g (or is preserved by g) if 
Z0 € Q for all 0 € Q, and if in addition for any 0^ € Q there exists 0 € Q 
such that 20 = 0’. These two conditions can be expressed by the equation 


(3) 72 - 9. 


The transformation g of € onto itself defined in this way is 1:1 provided 
the distributions P, corresponding to different values of @ are distinct. To 
see this let 70, = g6,. Then Pz9,(gA) = P59,( 8A) and therefore P4 (4) = 
Po (A) for all A, so that 6, = 4. 

Lemma 1. Let g,g' be two transformations preserving Q. Then the 
transformations g'g and g`" defined by 


(g'g)x = g(gx) andg(g x) -x forall xe 
also preserve Q and satisfy 


(4) gü-E.P and (67) -(D 


Proof. If the distribution of X is Pp, then that of gX is Pz and that of 
g’gX = g'(gX) is therefore Py s. This establishes the first equation of (4); 
the proof of the second one is analogous. 

We shall say that the problem of testing H: 0 € Qy against K : 6€, 
remains invariant under a transformation g if g preserves both Qy and Qg, 
so that the equation 


(5) Ely = Qu 
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holds in addition to (3). Let € be a class of transformations satisfying these 
two conditions, and let G be the smallest class of transformations contain- 
ing € and such that g, g' € G implies that g'g and g^! belong to G. Then 
G is a group of transformations, all of which by Lemma 1 preserve both Q 
and Qy. Any class € of transformations leaving the problem invariant can 
therefore be extended to a group G. It follows further from Lemma 1 that 
the class of induced transformations g form a group G. The two equations 
(4) express the fact that G is a homomorphism of G. 

In the presence of symmetries in both sample and parameter space 
represented by the groups G and G, it is natural to restrict attention to tests 
$ which are also symmetric, that is, which satisfy 


(6) e(gx)- (x) forall xe X and ge G. 


A test $ satisfying (6) is said to be invariant under G. The restriction to 
invariant tests is a particular case of the principle of invariance formulated 
in Section 5 of Chapter 1. As was indicated there and in the examples 
above, a transformation g can be interpreted as a change of coordinates. 
From this point of view, a test is invariant if it is independent of the 
particular coordinate system in which the data are expressed. 

A transformation g, in order to leave a problem invariant, must in 
particular preserve the class .»/ of measurable sets over which the distribu- 
tions P, are defined. This means that any set A € is transformed into a 
set of X and is the image of such a set, so that gA and g^ !4 both belong to 
wf. Any transformation satisfying this condition is said to be bimeasurable. 
Since a group with each element g also contains g !, its elements are 
automatically bimeasurable if all of them are measurable. If g^ and g are 
bimeasurable, so are g'g and g-!. The transformations of the group G 
above generated by a class € are therefore all bimeasurable provided this is 
the case for the transformations of €. 


2. MAXIMAL INVARIANTS 


If a problem is invariant under a group of transformations, the principle of 
invariance restricts attention to invariant tests. In order to obtain the best of 
these, it is convenient first to characterize the totality of invariant tests. 

Let two points x;, x; be considered equivalent under G, 


X; ~ x; (mod G), 


if there exists a transformation g € G for which x, = gx,. This is a true 
equivalence relation, since G is a group and the sets of equivalent points, 
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the orbits of G, therefore constitute a partition of the sample space. (Cf. 
Appendix, Section 1.) A point x traces out an orbit as all transformations g 
of G are applied to it; this means that the orbit containing x consists of the 
totality of points gx with g € G. It follows from the definition of invariance 
that a function is invariant if and only if it is constant on each orbit. 

A function M is said to be maximal invariant if it is invariant and if 


(7) M(x,) 2 M(x;) implies x, = gx, forsome g € G, 


that is, if it is constant on the orbits but for each orbit takes on a different 
value. All maximal invariants are equivalent in the sense that their sets of 
constancy coincide. 


Theorem 1. Let M(x) be a maximal invariant with respect to G. Then a 
necessary and sufficient condition for $ to be invariant is that it depends on x 
only through M(x), that is that there exists a function h for which $(x) = 
h[M(x)] for all x. 


Proof. If (x) = h[M(x)] for all x, then (gx) = h[M(gx)] = 
h{ M(x)] = (x) so that ¢ is invariant. On the other hand, if $ is invariant 
and if M(x,) = M(x), then x, = gx, for some g and therefore $(x;) = 
(xı). 


Example 1. (i) Let x = (Xp. Xn) and let G be the group of translations 
g(a bec) cone C= 00; 


Then the set of differences y = (Xi — Xps+++»%n-1 7 Xn) 18 invariant under G. To 
see that it is maximal invariant suppose that x, — x, = X, — Xn for di dose mds 
Putting x — x, =c, one has xj =x; + € for all i, as was to be shown. The 
function y is of course only one representation of the maximal invariant. Others are 
for example (x, — X2, X2 — X3- --s Xn-1 — Xn) OF the redundant (x — X,..., X, 
— X). In the particular case that n = 1, there are no invariants. The whole space is a 
single orbit, so that for any two points there exists a transformation of G taking one 
into the other. In such a case the transformation group G is said to be transitive. 
The only invariant functions are then the constant functions (x) = c. 


Gi) if G is the group of transformations 
gx = (035. 08) c * 0, 


a special role is played by any zero coordinates. However, in statistical applications 
the set of Rai which pe des of the coordinates is zero typically has probability 
1; attention can then be restricted to this part of the sample space, and the ggt of 
ratios xj/x,,..., X, 1 /X, i$ à i invariant. Without this restriction, two 
points x, x’ are equivalent with respect to the maximal invariant partition if among 
their coordinates there are the same number of zeros (if any), if these occur at the 
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same places, and if for any two nonzero coordinates x,, x, the ratios x;/x, and 
xj/x| are equal. 

(iii) Let x (x... x,) and let G be the group of all orthogonal transforma- 
tions x’ = Tx of n-space. Then Zx? is maximal invariant, that is, two points x and 
x* can be transformed into each other by an orthogonal transformation if and only 
if they have the same distance from the origin. The proof of this is immediate if one 
restricts attention to the plane containing the points x, x* and the origin. 


Example 2. (i) Let x= (x3,..., Xn) and let G be the set of n! permutations 
of the coordinates of x. Then the set of ordered coordinates (order statistics) 
Xa) € 5 < x is maximal invariant. A permutation of the x; obviously does not 
change the set of values of the coordinates and therefore not the xq). On the other 
hand, two points with the same set of ordered coordinates can be obtained from 
each other through a permutation of coordinates. 

(ii) Let G be the totality of transformations x; = f(x;), i= 1,..., n, such that 
f is continuous and strictly increasing, and suppose that attention can be restricted 
to the points all of whose n coordinates are distinct. If the x, are considered as n 
points on the real line, any such transformation preserves their order. Conversely, if 
Xj... X, and x[,..., x/ are two sets of points in the same order, say x, < ++: < 
x, and x, < +++ < xj, there exists a transformation f satisfying the required 
conditions and such that x/ = f(x) for all i. It can be defined for example as 
f(x) =x + (xj, — x) for x < xi, f(x) = x + (xi, — xi) for x > x;, and to be 
linear between x, and x, for k — 1,...,n — T. A formal expression for the 
maximal invariant in this case is the set of ranks (r,,..., 7,) of (xy,..., Xn). Here 
the rank r, of x, is defined through 


Hi = XQ) 


so that r, is the number of x’s < x. In particular r; = 1 if x, is the smallest x, 
r; = 2 if it is the second smallest, and so on. 


Example 3. Let x be an n X s matrix (s < n) of rank s, and let G be the group 
of linear transformations gx = xB, where B is any nonsingular s X s matrix. Then 
a maximal invariant under G is the matrix t(x) = x(x’x)~'x’, where x’ denotes the 
transpose of x. Here (x'x)' ! is meaningful because the s x s matrix x/x is 


nonsingular; in fact, it will be shown in Lemma 1 of Chapter 8 that x’x is positive 
efinite. 


That t(x) is invariant is clear, since 
t( gx) = xB(B'x'xB) !B'x' = x(x'x) x = t(x). 
To see that 1(x) is maximal invariant, suppose that 


» 


xx) xt F xal 24x2) x2- 


f s y : E 
Since (x[x;)"! is positive definite, there exist nonsingular matrices C; such that 
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(xix)! = C,C/ and hence 
(4Q,)(4G) = (4G) (2G): 


As will be shown in Chapter 8, Section 2, this implies the existence of an orthogonal 
matrix Q such that x,C, = x,C,Q and thus x; = x,B with B = C,QC;', as was to 
be shown. 

In the special case s = n, we have t(x) = J, so that there are no nontrivial 
invariants. This corresponds to the fact that in this case G is transitive, since any 
two nonsingular n X n matrices x, and x, satisfy x, = xB with B = x; 1x;. 

This result can be made more intuitive through a geometric interpretation. 
Consider the s-dimensional subspace S of R” spanned by the s columns of x. Then 
P = x(x'x) x has the property that for any y in R”, the vector Py is the 
projection of y onto S. (This will be proved in Chapter 7, Section 2.) The invariance 
of P expresses the fact that the projection of y onto S is independent of the choice 
of vectors spanning S. To see that it is maximal invariant, suppose that the 
projection of every y onto the spaces S, and S, spanned by two different sets of s 
vectors is the same. Then S, = Sj, so that the two sets of vectors span the same 
space. There then exists a nonsingular transformation taking one of these sets into 
the other. 


A somewhat more systematic way of determining maximal invariants is 
obtained by selecting, by means of a specified rule, a unique point M(x) on 
each orbit. Then clearly M(X) is maximal invariant. To illustrate this 
method, consider once more two of the earlier examples. 


Example 1(i) (continued). The orbit containing the point (a,,..., 4,) under the 
group of translations is the set {(a, + 6... 44 + €), =% <¢< oo), which is a 
line in E,. 


(a) As representative point M(x) on this line, take its intersection with the 
hyperplane x, = 0. Since then a, + c — 0, this point corresponds to the value 
c= —a, and thus has coordinates (2; — 4n». --» 05-17 a,,0). This leads to the 
maximal invariant (Xj — Xn» -- +» Xn-1 — Xn) 

(b) An alternative point on the line is its intersection with the hyperplane 
Ex, = 0. Then c = —a, and M(a) = (a — à... än — 5). , i 

c) The point need not be ified by an intersection property. It can for 
iiie be taken as the point on the line that is closest to the origin. Since the value 
of c minimizing D(a, + c)? is c= —ā, this leads to the same point as (b). 


Example 1 (iii) (continued). The orbit containing the point (a,,..-, a,,) under the 
group of orthogonal transformations is the hypersphere containing (4),..-, a„) and 
with center at the origin. As representative point on this sphere, take its north pole, 
ie. the point with a, = ~ =a,- 7 0. The coordinates of this point are 
(0,....0, Za? ) and hence lead to the maximal invariant Dx/. (Note that in this 
example, the determination of the orbit is essentially equivalent to the determination 
of the maximal invariant.) 


Frequently, it is convenient to obtain a maximal invariant in a number of 
f G. To illustrate the process and 


steps, each corresponding to a subgroup ©! 
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a difficulty that may arise in its application, let x = (xy,..., x,), suppose 
that the coordinates are distinct, and consider the group of transformations 


gx = (ax, * b,...,ax,* b), a*0, —oo « b « co. 


Applying first the subgroup of translations x/ = x; b, a maximal in- 
variant is y = (yy... Yn-1) With y; = x; — Xn Another subgroup consists 
of the scale changes x/^ = ax;. This induces a corresponding change of scale 
in the y’s: y/’ = ay and a maximal invariant with respect to this group 
acting on the y-space is z = (2),-.-,Z,—-2) with z; = ¥i/¥n—1- Expressing 
this in terms of the x’s, we get z; = (x, — x,)/(x,-1 — Xn)» Which is 
maximal invariant with respect to G. 

Suppose now the process is carried out in the reverse order. Application 
first of the subgroup x/ = ax, yields as maximal invariant u = 
(us .., 4,1) With u; = x,/x,. However, the translations x/ = x, + b do 
not induce transformations in u-space, since (x; + b)/(x, + b) is not a 
function of x,/x,. 

Quite generally, let a transformation group G be generated by two 
subgroups D and E in the sense that it is the smallest group containing D 
and E. Then G consists of the totality of products e,d,,...e,d for 
m — 1,2,..., with d; € D, e, € E(i = 1,..., m)! The following theorem 
shows that whenever the process of determining a maximal invariant in 
iue can be carried out at all, it leads to a maximal invariant with respect 
to G. 


Theorem 2. Let G be a group of transformations, and let D and E be two 
subgroups generating G. Suppose that y — s(x) is maximal invariant with 
respect to D, and that for any e € E 
(8) s(x,) 9 s(x3) implies s(ex,) = s(ex;). 


If z — t(y) is maximal invariant under the group E* of transformations e* 
defined by 


e*y=s(ex) when y= s(x), 
then z = t[s(x)] is maximal invariant with respect to G. 
Proof. To show that #[s(x)] is invariant, let x’ = gx, g = emd m- 6141: 
Then 
t[s(x’)] =2[s(e,,d,,---€14,x)] = t[ets(d,, ...e1d1x)] 
= t[s(e, —1d m1 --- 61dix)]; 


*See Section 1 of the Appendix. 
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and the last expression can be reduced by induction to r[s(x)]. To see that 
t[s(x)] is in fact maximal invariant, suppose that 1[s(x^)] = t[s(x)]. Setting 
y'= s(x’), y s(x), one has t(y’) = t(y), and since t(y) is maximal 
invariant with respect to E*, there exists e* such that y’ = e*y. Then 
s(x’) = e*s(x) = s(ex), and by the maximal invariance of s(x) with respect 
to D there exists d € D such that x’ = dex. Since de is an element of G 
this completes the proof. 

Techniques for obtaining the distribution of maximal invariants are 
discussed by Andersson (1982), Eaton (1983), Farrell (1985), and Wijsman 
(1985). 


3. MOST POWERFUL INVARIANT TESTS 


The class of all invariant functions can be obtained as the totality of 
functions of a maximal invariant M(x). Therefore, in particular the class of 
all invariant tests is the totality of tests depending only on the maximal 
invariant statistic M. The latter statement, while correct for all the usual 
situations, actually requires certain qualifications regarding the class of 
measurable sets in M-space. These conditions will be discussed at the end of 
the section; they are satisfied in the examples below. 

Example 4. Let X —(X,,... Xn)» and suppose that the density of X is 
f(x, — 0,..., x, — 0) under H, (i = 0,1), where 0 ranges from —oo to oo. The 
problem of testing Ho against H; is invariant under the group G of transformations 


gx- (x Hep.) x, Fe) I So ce o, 
which in the parameter space induces the transformations 
gü -0- c. 


By Example 1, a maximal invariant under G is Y — (Xi 7 X, XS 7 Xn). The 
distribution of Y is independent of @ and under H; has the density 


JE f t 2e 3 z,z) dz. 
-%0 


When referred to Y, the problem of testing Hp against H, therefore becomes one of 
testing a simple hypothesis against a simple alternative. The most powerful test is 
then independent of 0, and therefore UMP among all invariant tests. Its rejection 
region by the Neyman-Pearson lemma is 


'oo ‘CO 
f fO t nest 252) dz ji Ai + us., Xn + U) du 
EN -0 


Sid mosa e adn cad; AA 
J fn + ze» * z,2)d: [fe ases Xa + U) du 
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A general theory of separate families of hypotheses (in which the family K 
of alternatives does not adjoin the hypothesis H but, as in Example 4, is 
separated from it) was initiated by Cox (1961, 1962). A bibliography of the 
subject is given in Pereira (1977); see also Loh (1985). 

Before applying invariance, it is frequently convenient first to reduce the 
data to a sufficient statistic T. If there exists a test (T) that is UMP 
among all invariant tests depending only on T, one would like to be able to 
conclude that $ọ(T) is also UMP among all invariant tests based on the 
original X. Unfortunately, this does not follow, since it is not clear that for 
any invariant test based on X there exists an equivalent test based on T, 
which is also invariant. Sufficient conditions for ġọ(T ) to have this property 
are provided by Hall, Wijsman, and Ghosh (1965) and Hooper (1982a), and 
a simple version of such a result (applicable to Examples 5 and 6 below) will 
be given by Theorem 6 in Section 5. The relationship between sufficiency 
and invariance is discussed further in Berk (1972) and Landers and Rogge 
(1973). 


Example 5. If X,,..., X, is a sample from N(£, o°), the hypothesis H : o > % 
remains invariant under the transformations X/ = X, + c, —oo < c < oo. In terms 
of the sufficient statistics Y = X, S? = E(X; — X)? these transformations become 
Y’ = Y + c, (S? = S?, and a maximal invariant is S*. The class of invariant tests 
is therefore the class of tests depending on S?. It follows from Theorem 2 of 
Chapter 3 that there exists a UMP invariant test, with rejection region E(X; — X y 
< C. This coincides with the UMP unbiased test (9) of Chapter 5. 


Example 6. If X,..., X, and Y,...,Y, are samples from. N(£, o?) and 
N(n, 12), a set of sufficient statistics is T, = X, T; = Y, T, = VE(% — X) , and 
T, = yX(Y, - Y). The problem of testing H:72/o? < Ao remains invariant 
under the transformations T; = T, + c, T} = T; + c, Tj = Tj, Ti = T4, —© < 
€, €; < œ, and also under a common change of scale of all four variables. A 
maximal invariant with respect to the first group is (7;, T4). In the space of this 
maximal invariant, the group of scale changes induces the transformations Ty’ = cT, 
Tj’ = en, 0 < c, which has as maximal invariant the ratio T,/T,. The statistic 
Z = (T2/(n — 1)] = [T?/(m — 1)] on division by A = 2/6? has an F-distribution 
with density given by (21) of Chapter 5, so that the density of Z is 


c(A) 80-9 


py imen-2)' 
(a+ r ) 


mia 


z>0. 


For varying A, these densities constitute a family with monotone likelihood ratio, so 
that among all tests of H based on Z, and therefore among all invariant tests, there 


exists a UMP one given by the rejection region Z > C. This coincides with the 
UMP unbiased test (20) of Chapter 5. = ee 
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Example 7. In the method of paired comparisons for testing whether a treat- 
ment has a beneficial effect, the experimental material consists of m pairs of 
subjects. From each pair, a subject is selected at random for treatment while the 
other serves as control. Let X, be 1 or 0 as for the ith pair the experiment turns out 
in favor of the treated subject or the control and let p, = P{X; = 1). The 
hypothesis of no effect, H: p, = l for i-1...,n, is to be tested against the 
alternatives that p, > } for all i. 

The problem remains invariant under all permutations of the m variables 
X,,..., X,, and a maximal invariant under this group is the total number of 


n? 


successes X = X, + -:- +X,- The distribution of X is 


Pi Pi, 
P(X =k} = Qe Goo 
{ =h- A 


where q; =1-— p; and where the summation extends over all () choices of 
subscripts i, < ::* < ię- The most powerful invariant test against an alternative 
(p^... Pi) rejects H when 


To Sia eel 
pum rie 


c 


To see that f is an increasing function of k, note that a, = p//4 > 1, and that 


Y Xaja,...a,7 (k* 1) Daj, 44, 
jJ 


and 


YXa,..a,7(- k) Ea, tn 
Jj 


Here, in both equations, the second summation on the left-hand side extends over 
all subscripts i, < +++ < i, of which none is equal to j, and the summation on the 
right-hand side extends over all subscripts i < ~~ «i,,; and ij € 1s € iy 
respectively without restriction. Then 


k+1 d 


1 1 
"rr gut on as 244.) 
f(k +1) ( ii yee lipsi e-o)? aja a 


1 
= Tay Lh, diy = f(k), 
B 
as was to be shown. Regardless of the alternative chosen, the test therefore rejects 
when k > C, and hence is UMP invariant. If the ith comparison is considered plus 


nn 
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or minus as X, is 1 or 0, this is seen to be another example of the sign test. (Cf. 
Chapter 3, Example 8, and Chapter 4, Section 9.) 


Sufficient statistics provide a simplification of a problem by reducing the 
sample space; this process involves no change in the parameter space. 
Invariance, on the other hand, by reducing the data to a maximal invariant 
statistic M, whose distribution may depend only on a function of the 
parameter, typically also shrinks the parameter space. The details are given 
in the following theorem. 


Theorem 3. If M(x) is invariant under G, and if v(0) is maximal 
invariant under the induced group G, then the distribution of M( X) depends 
only on v(0). 


Proof. Let v(@,) = v(0,). Then 6; = g0,, and hence 
P,(M(X) e€Bj- Pag ( M(X) €Bj- P,,{ M(gX) € B) 
= P,(M(X) € B). 


This result can be paraphrased by saying that the principle of invariance 
identifies all parameter points that are equivalent with respect to G. 

In application, for instance in Examples 5 and 6, the maximal invariants 
M(x) and 6 = v(0) under G and G are frequently real-valued, and the 
family of probability densities p,(m) of M has monotone likelihood ratio. 
For testing the hypothesis H : 8 < 8, there exists then a UMP test among 
those depending only on M, and hence a UMP invariant test. Its rejection 
region is M 7 C, where 


(9) f "pa (m) dm = a. 


Consider this problem now as a two-decision problem with decisions do 
and d, of accepting or rejecting H, and a loss function L(@, d;) = L;(@). 
Suppose that L;(0) depends only on the parameter ô, L,(@) = L/(6) say, 
and satisfies 


(10) L(8)-L(8)Z0 as 855. 


It then follows from Theorem 3 of Chapter 3 that the family of rejection 
regions M > C(a), as a varies from 0 to 1, forms a complete family of 
decision procedures among those depending only on M, and hence a 
complete family of invariant procedures. As before, the choice of a particu- 


lar significance level a can be considered as a convenient way of specifying 
a test from this family. 
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At the beginning of the section it was stated that the class of invariant 
tests coincides with the class of tests based on a maximal invariant statistic 
M = M( X). However, a statistic is not completely specified by a function, 
but requires also specification of a class @ of measurable sets. If in the 
present case 4 is the class of all sets B for which M^ (B) € æ, the desired 
statement is correct. For let $(x) = y[M(x)] and $ by measurable, and 
let C be a Borel set on the line. Then $ (C) = M^ '[y(C)] ©. and 
hence ij^ (C) € 2, so that y is @measurable and $(x) = V[M(x)] is a 
test based on the statistic M. 

In most applications, M(x) is a measurable function taking on values in 
a Euclidean space and it is convenient to take Z as the class of Borel sets. If 
(x) = V[M(x)] is then an arbitrary measurable function depending only 
on M(x), it is not clear that (m) is necessarily measurable. This 
measurability can be concluded if Z is also Euclidean with s the class of 
Borel sets, and if the range of M is a Borel set. We shall prove it here only 
under the additional assumption (which in applications is usually obvious, 
and which will not be verified explicitly in each case) that there exists a 
vector-valued Borel-measurable function Y(x) such that [M(x), Y(x)] maps 
4 onto a Borel subset of the product space .A X Y, that this mapping is 
1:1, and that the inverse mapping is also Borel-measurable. Given any 
measurable function $ of x, there exists then a measurable function ¢ of 
(m, y) such that $(x) = ¢[M(x), Y(x)). If $ depends only on M(x), then 
4/ depends only on m, so that d/(m, y) = v (m) say, and y is a measurable 


function of m.* In Example 1(i) for instance, where x = Gtr, see ON) 
and M(x) = (x, — x,,..., X4 1 — Xn)» the function Y(x) can be taken as 
Y(x) 7 x,. 


4. SAMPLE INSPECTION BY VARIABLES 


A sample is drawn from a lot of some manufactured product in order to 
decide whether the lot is of acceptable quality. In the simplest case, each 
sample item is classified directly as satisfactory or defective (inspection by 
attributes), and the decision is based on the total number of defectives. 
More generally, the quality of an item is characterized by a variable Y 
(inspection by variables), and an item is considered satisfactory if Y exceeds 
a given constant u. The probability of a defective is then 


p=P{Y <u} 
and the problem becomes that of testing the hypothesis H : p > Po. 


*The last statement is an immediate consequence, for example, of Theorem B, Section 34, 
of Halmos (1974). 
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As was seen in Example 8 of Chapter 3, no use can be made of the actual 
value of Y unless something is known concerning the distribution of Y. In 
the absence of such information, the decision will be based, as before, 
simply on the number of defectives in the sample. We shall consider the 
problem now under the assumption that the measurements Yj,..., Y, con- 
stitute a sample from N(n, o°). Then 


p= fme ao = y dy = a = +), 


2mo c 


where 
y 1 
- — -1p] 
(y) Í ge” 2t ) dt 


denotes the cumulative distribution function of a standard normal distribu- 
tion, and the hypothesis H becomes (u — )/o > 9^ ( po). In terms of the 
variables X, = Y, — u, which have mean £ =  — u and variance c?, this 
reduces to 


£ 
Hi 
PELO 


with 6, = —®~1( po). This hypothesis, which was considered in Chapter 5 
Section 2, for 6, = 0, occurs also in other contexts. It is appropriate when 
one is interested in the mean £ of a normal distribution, expressed in o-units 
rather than on a fixed scale. 


For testing H, attention can be restricted to the pair of variables X and 
S-yx(X-X 0s , since they form a set of sufficient statistics for (£, o), 
which satisfy the conditions of Theorem 6 of the next section. These 
variables are independent, the distribution of X being N(£, o?/n) and that 
of S/o being x, ;. Multiplication of X and S by a common constant 
c > 0 transforms the parameters into ¢’ = c£, o’ = co, so that £/o and 
hence the problem of testing H remain invariant. A maximal invariant 
under these transformations is X/s or 


Ynx 
s/i(n-1"' 


the distribution of which depends only on the maximal invariant in the 
parameter space 0 = £/6 (cf. Chapter 5, Section 2). Thus, the invariant tests 
are those depending only on t, and it remains to find the most powerful test 
of H: 0 < 0, within this class. 


t= 
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The probability density of t is (Chapter 5, Problem 3) 


oo 1 w 2 T 
P(t) = cf zs al ght J [ieena dw, 


where 5 = vn 0 is the noncentrality parameter, and this will now be shown 
to constitute a family with monotone likelihood ratio. To see that the ratio 


oo 1 w 2 1 
[ex st XI -4| wi Pexp(— jw) dw 
o CE 
"m 7 oo 1 LA f 
Eel jene e eet Ln 2exp( — 1 
fk exp zl m a) wi"exp(— lw) dw 


is an increasing function of t for 8) < ô, suppose first that t < 0 and let 
v = —t/w/(n — 1). The ratio then becomes proportional to 


00 md v? 
f Joal- - &)o- tepe] do 


[roe zen, 


= fex - (8: - 9)v] geo) do 


where 
f(v) = expl- ôw) v 'exp( -*/2) 
and 
y 2 
foja- gos 
ga(v) = (n = D2 


[feot- 32 |* 


Since the family of probability densities g,2(v) is a family with monotone 
likelihood ratio, the integral of expl- (8, — 5)2] with respect to this 
density is a decreasing function of t? (Problem 14 of Chapter 3), and hence 
an increasing function of t for t < 0. Similarly one finds that r(t) is an 
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increasing function of t for t>0 by making the transformation v = 
t(w/(n — 1) . By continuity it is then an increasing function of t for all z. 

There exists therefore a UMP invariant test of H: £/o < bo, which 
rejects when t > C, where C is determined by (9). In terms of the original 
variables Y, the rejection region of the UMP invariant test of H: p > po 
becomes 


vn (F - u) 


(11) 
VX(4- 5) An - 1) 


If the problem is considered as a two-decision problem with losses Lo( p) 
and L,(p) for accepting or rejecting p > po, which depend only on p and 
satisfy the condition corresponding to (10), the class of tests (11) constitutes 
a complete family of invariant procedures as C varies from — oo to oo. 

Consider next the comparison of two products on the basis of samples 
X,,..., Xm) Y... Y, from N(£, o?) and N(n, o°). If 


reels ee) 


o 


SiGe 


one wishes to test the hypothesis p < 7, which is equivalent to 


H:n<é. 


The statistics X, Y, and $ = yxGx; = Mok r(Y, = Y) are a set of 
sufficient statistics for £, v, o. The problem remains invariant under the 
addition of an arbitrary common constant to X and Y, which leaves Y — X 
and S as maximal invariants. It is also invariant under multiplication of X, 
Y, and S, and hence of Y — X and S, by a common positive constant, which 
reduces the data to the maximal invariant (Y — X)/S. Since 


(»- x)» a aba 
dis s/im *n-2 


has a noncentral #-distribution with noncentrality parameter ô = vn (n 
-ty/ vm + no, the UMP invariant test of H:n — £ < 0 rejects when 
t > C. This coincides with the UMP unbiased test (27) of Chapter 5, Section 
3. Analogously, the corresponding two-sided test (30) of Chapter 5, with 
rejection region |t| = C, is UMP invariant for testing the hypothesis p = 7 
against the alternatives p + v (Problem 9). 
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5. ALMOST INVARIANCE 


Let G be a group of transformations leaving a family 2 = { P}, 0 € Q} of 
distributions of X invariant. A test $ is said to be equivalent to an invariant 
test if there exists an invariant test p such that $(x) = (x) for all x 
except possibly on a Z-null set N; $ is said to be almost invariant with 
respect to G if 


(12) (gx) -é(x) forall xeX-N, geG 


where the exceptional null set N, is permitted to depend on g. This concept 
is required for investigating the relationship of invariance to unbiasedness 
and to certain other desirable properties. In this connection it is important 
to know whether a UMP invariant test is also UMP among almost invariant 
tests. This turns out to be the case under assumptions which are made 
precise in Theorem 4 below and which are satisfied in all the usual 
applications. 

If $ is equivalent to an invariant test, then $(gx) = $(x) for all 
x € NU g-!N. Since P(g !N) = P(N) = 0, it follows that $ is then 
almost invariant. The following theorem gives conditions under which 
conversely any almost invariant test is equivalent to an invariant one. 


Theorem 4. Let G be a group of transformations of 2, and let x and B 
be o-fields of subsets of X and G such that for any set A € s the set of pairs 
(x, g) for which gx € A is measurable xf X B. Suppose further that there 
exists a o-finite measure v over G such that v(B) = 0 implies v(Bg) = 0 for 
all g € G. Then any measurable function that is almost invariant under G 
(where “almost” refers to some a-finite measure p) is equivalent to an 
invariant function. 

Proof. Because of the measurability assumptions, the function $(gx) 
considered as a function of the two variables x and g is measurable 2 X B. 
It follows that $(gx) — (x) is measurable s X 4, and so therefore is the 
set S of points (x, g) with (gx) + (2). If @ is almost invariant, any 
section of S with fixed g is a p-null set. By Fubini’s theorem (Theorem 3 of 
Chapter 2) there exists therefore a p-null set N such that for all x € X- N 


olg) = 9(x) aer. 


Without loss of generality suppose that »(G) = 1, and let A be the set of 
points x for which 


fole) dlg) =l) ae» 
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If 


f(x, 8) =| feta) dva?) - 68), 
then A is the set of points x for which 


[fG 8) dv(g) = 0. 


Since this integral is a measurable function of x, it follows that A is 
measurable. Let 


W(x) = Jo(sx) avg) if x€A, 
0 if x€A. 


Then y is measurable and y(x) = (x) for x € N, since $(gx) = $(x) 
a.e. v implies that {(g’x) dv(g’) = (x) and that x € A. To show that V 
is invariant it is enough to prove that the set A is invariant. For any point 
x € A, the function $(gx) is constant except on a null subset N, of G. Then 
$(ghx) has the same constant value for all g ¢ N,h~!, which by assump- 
tion is again a »-null set; and hence hx € A, which completes the proof. 
Additional results concerning the relation of invariance and almost 
invariance are given by Berk and Bickel (1968) and Berk (1970). In 
particular, the basic idea of the following example is due to Berk (1970). 


Example 8. Counterexample. Let Z,Y,,..., Y, be independently distributed as 
N(6,1), and consider the 1:1 transformations y/ = y, (i = 1,..., n) and 


Z'-— except for a finite number of points a,,..., a, for 
which a; = a, for some permutation (j,,..., jk) of (1,..., k). 


If the group G is generated by taking for (a,,...,a,), k = 1,2,..., all finite sets 
and for (j;,..., ję) all permutations of (1,..., k), then (z, y;,.-.,;,) is almost 
invariant. It is however not equivalent to an invariant function, since (),,--- Yn) i8 
maximal invariant. 


Corollary 1. Suppose that the problem of testing H : 0 € w against K : 0 


€ Q — w remains invariant under G and that the assumptions of Theorem 4 


hold. Then if $9 is UMP invariant, it is also UMP within the class of almost 
invariant tests. 
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Proof. If $ is almost invariant, it is equivalent to an invariant test ij by 
Theorem 4. The tests ó and y have the same power function, and hence pọ 
is uniformly at least as powerful as 4. 

In applications, 2? is usually a dominated family, and p any o-finite 
measure equivalent to P (which exists by Theorem 2 of the Appendix). If $ 
is almost invariant with respect to 2, it is then almost invariant with respect 
to p. and hence equivalent to an invariant test. Typically, the sample space 4^ 
is an n-dimensional Euclidean space, is the class of Borel sets, and the 
elements of G are transformations of the form y = f(x, 7), where 7 ranges 
over a set of positive measure in an m-dimensional space and f is a 
Borel-measureable vector-valued function of m + n variables. If 2 is taken 
as the class of Borel sets in m-space, the measurability conditions of the 
theorem are satisfied. 

The requirement that for all g € G and B € 4 


(13) »(B)-0 implies »(Bg) =0 
is satisfied in particular when 
(14) »(Bg)=»(B) forall geG, BEZ. 


The existence of such a right invariant measure is guaranteed for a large 
class of groups by the theory of Haar measure. Alternatively, it is usually 
not difficult to check the condition (13) directly. 


Example 9. Let G be the group of all nonsingular linear transformations of 
n-space. Relative to a fixed coordinate system the elements of G can be represented 
by nonsingular n x n matrices A = (a,;), 4’ = (aij) --- with the matrix product 
serving as the group product of two such elements. The o-field 2 can be taken to be 
the class of Borel sets in the space of the n^ elements of the matrices, and the 
measure » can be taken as Lebesgue measure over 2. Consider now a set S of 
matrices with v(S) = 0, and the set S* of matrices A'A with ^ [ S and A fixed. 
If a = max|a;|, C' = A’A, and C" = AA, the inequalities |2/; — alse for all 
i, j imply |c}; — cj, < nae. Since a set has »-measure zero if and only if it can be 
covered by a union of rectangles whose total measure does not exceed any given 
€ > 0, it follows that v(S*) = 0, as was to be proved. 


In the preceding chapters, tests were compared purely in terms of their 
power functions (possibly weighted according to the seriousness of the 
losses involved). Since the restriction to invariant tests 1s a departure from 
this point of view, it is of interest to consider the implications of applying 
invariance to the power functions rather than to the tests themselves. Any 
test that is invariant or almost invariant under a group G has a power 


function which is invariant under the group G induced by G in the 
parameter space. 
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To see that the converse is in general not true, let Xj, X,, X, be 
independently, normally distributed with mean £ and variance o°, and 
consider the hypothesis o > a. The test with rejection region 


|X,- X|»k when X«0, 
|X,- Xj»k when X20 


is not invariant under the group G of transformations X/ = X; + c, but its 
power function is invariant under the associated group G. 

The two properties, almost invariance of a test ọ and invariance of its 
power function, become equivalent if before the application of invariance 
considerations the problem is reduced to a sufficient statistic whose distribu- 
tions constitute a boundedly complete family. 


Lemma 2. Let the family PT = (Pj, 0 € Q} of distributions of T be 
boundedly complete, and let the problem of testing H : 0 € Qy remain in- 
variant under a group G of transformations of T. Then a necessary and 
sufficient condition for the power function of a test (t) to be invariant under 
the induced group G over Q is that W(t) is almost invariant under G. 


Proof. For all € € Q we have E;jJ(T) = Egy(gT). If } is almost 
invariant, Eg} (T) = Eyy(gT) and hence EY (T) = Ep} (T), so that the 
power function of y is invariant. Conversely, if Ej (T) = Ezg} (T), then 
EJ (T) = Eo} (gT), and it follows from the bounded completeness of pT 
that (gt) = (t) ae. 27. 

As a consequence, it is seen that UMP almost invariant tests also possess 
the following optimum property. 


Theorem 5. Under the assumptions of Lemma 2, let v(@) be maximal 
invariant with respect to G, and suppose that among the tests of H based on 
the sufficient statistic T there exists a UMP almost invariant one, say Yo(t). 
Then y(t) is UMP in the class of all tests based on the original observations 
X, whose power function depends only on v(@). 


Proof. Let (x) be any such test, and let y(t) = E[9(X)|r]. The 
power function of (1), being identical with that of $(x), depends then 
only on v(0), and hence is invariant under G. It follows from Lemma 2 that 
y(t) is almost invariant under G, and y(t) is uniformly at least as 
powerful as y(t) and therefore as $(x). 


Example 10. For the hypothesis 7? < o? concerning the variances of two 
normal distributions, the statistics (X, Y, S2, S2) constitute a complete set of 
sufficient statistics. It was shown in Example 6 that there exists a UMP invariant 
test with respect to a suitable group G, which has rejection region S?/S% > Co- 
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Since in the present case almost invariance of a test with respect to G implies that it 
is equivalent to an invariant one (Problem 12), Theorem 5 is applicable with 
v(0) = A = 1?/o?, and the test is therefore UMP among all tests whose power 
function depends only on A. 


Theorem 4 makes it possible to establish a simple condition under which 
reduction to sufficiency before the application of invariance is legitimate. 


Theorem 6. Let X be distributed according to P}, 0 € Q, and let T be 
sufficient for 0. Suppose G leaves invariant the problem of testing H: 0 € Qy 
and that T satisfies 


T(x,) 2 T(x) implies T(gx,) ^ T(gx;) forall g € G, 
so that G induces a group G of transformations of T-space through 
&T(x) = T(gx). 


G) If p(x) is any invariant test of H, there exists an almost invariant test 
wW based on T, which has the same power function as q. 3 

(i) If in addition the assumptions of Theorem 4 are satisfied, the test Y 
of (i) can be taken to be invariant. à 

Gii) If there exists a test (T) which is UMP among all G-invariant 
tests based on T, then under the assumptions of (ii), Yo is also UMP among 
all G-invariant tests based on X. 

This theorem justifies the derivation of the UMP invariant tests of 
Examples 5 and 6. 

Proof. (i): Let p(t) = E[p(X)|t]. Then y has the same power function 
as p. To complete the proof, it suffices to show that y(t) is almost 
invariant, i.e. that 


v(a)=v(t) (a.e. 27). 
It follows from (1) that 
E,[e(gX)]g] = Eg[eCOl] — (e. Po). 


Since T is sufficient, both sides of this equation are independent of 0. 
Furthermore p(gx) = p(x) for all x and g, and this completes the proof. 
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Part (ii) follows immediately from (i) and Theorem 4, and part (iii) from 


(ii). 
6. UNBIASEDNESS AND INVARIANCE 


The principles of unbiasedness and invariance complement each other in 
that each is successful in cases where the other is not. For example, there 
exist UMP unbiased tests for the comparison of two binomial or Poisson 
distributions, problems to which invariance considerations are not applica- 
ble. UMP unbiased tests also exist for testing the hypothesis o = o, against 
c * c, in a normal distribution, while invariance does not reduce this 
problem sufficiently far. Conversely, there exist UMP invariant tests of 
hypotheses specifying the values of more than one parameter (to be consid- 
ered in Chapter 7) but for which the class of unbiased tests has no UMP 
member. There are also hypotheses, for example the one-sided hypothesis 
£/o < 0, in a univariate normal distribution or p < pọ in a bivariate one 
(Problem 10) with 6o, po # 0, where a UMP invariant test exists but the 
existence of a UMP unbiased test does not follow by the methods of 
Chapter 5 and is an open question. 

On the other hand, to some problems both principles have been applied 
successfully. These include Student's hypotheses £ < £y and = £o concern- 
ing the mean of a normal distribution, and the corresponding two-sample 
problems y — £ x Ay and 5$ — £ 2 A; when the variances of the two 
samples are assumed equal. Other examples are the one-sided hypotheses 
c? > ad and 72/o? > A, concerning the variances of one or two normal 
distributions. The hypothesis of independence p = 0 in a bivariate normal 
distribution is still another case in point (Problem 10). In all these examples 
the two optimum procedures coincide. We shall now show that this is not 
accidental but, is the case whenever the UMP invariant test is UMP also 
among all almost invariant tests and the UMP unbiased test is unique. In 
this sense, the principles of unbiasedness and of almost invariance are 
consistent. i 


Theorem 7. Suppose that for a given testing problem there exists a UMP 
unbiased test * which is unique (up to sets of measure zero), and that there 
also exists a UMP almost invariant test with respect to some group G. Then 


the latter is also unique (up to sets of measure zero), and the two tests coincide 
a.e. 


Proof. if U(a) is the class of unbiased level-a tests, and if g € G, then 
$ € U(a) if and only if og € U(a).t Denoting the power function of the 


tpg denotes the critical function which assigns to x the value $(gx). 
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test by B,(0), we thus have 


Bye g(8) = By» (g0) = sup B,(g8)— sup B,,(0) 
$€U(a) $€U(a) 


sup B,,(8) 2 By (8). 


$g€U(a) 


It follows that ¢* and $*g have the same power function, and, because of 
the uniqueness assumption, that $* is almost invariant. Therefore, if $^ is 
UMP almost invariant, we have 8,(8) > B+ (0) for all 0. On the other 
hand, $' is unbiased, as is seen by comparing it with the invariant test 
(x) = a, and hence 8,(0) < Bye(@) for all 6. Since 4/ and $* therefore 
have the same power function, they are equal a.e. because of the uniqueness 
of $*, as was to be proved. 

This theorem provides an alternative derivation for some of the tests of 
Chapter 5. In Theorem 3 of Chapter 4, the existence of UMP unbiased tests 
was established for one- and two-sided hypotheses concerning the parame- 
ter 0 of the exponential family (10) of Chapter 4. For this family, the 
statistics (U, T") are sufficient and complete, and in terms of these statistics 
the UMP unbiased test is therefore unique. Convenient explicit expressions 
for some of these tests, which were derived in Chapter 5, can instead be 
obtained by noting that when a UMP almost invariant test exists, the same 
test by Theorem 7 must also be UMP unbiased. This proves for example 
that the tests of Examples 5 and 6 of the present chapter are UMP 
unbiased. 

The principles of unbiasedness and invariance can be used to supplement 
each other in cases where neither principle alone leads to a solution but 
where they do so when applied in conjunction. As an example consider a 
sample X,,..., X, from N(£, o?) and the problem of testing H : £/o = 6, 
* 0 against the two-sided alternatives that £/o + 6). Here sufficiency 
and invariance reduce the problem to the consideration of t= Vn x/ 
yX(x,- x '/(n — 1). The distribution of this statistic is the noncentral 
t-distribution with noncentrality parameter ô = yn£/c and n — 1 degrees 
of freedom. For varying 8, the family of these distributions can be shown to 
be STP, [Karlin (1968, pp. 118-119; see Chapter 3, Problem 27] and hence 
in particular STP}. It follows by Problem 29 of Chapter 3 that among all 
tests of H based on f, there exists a UMP unbiased one with acceptance 
region C, < t < C,, where Cj, C, are determined by the conditions 


aP,{C, <t< C) 
pC et i aa ee 


P,{C,<t<G}=1-a and 28 T 
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In terms of the original observations, this test then has the property of being 
UMP among all tests that are unbiased and invariant. Whether it is also 
UMP unbiased without the restriction to invariant tests is an open problem. 

An analogous example occurs in the testing of the hypotheses H : p = po 
and H': p, < p < p; against two-sided alternatives on the basis of a sample 
from a bivariate normal distribution with correlation coefficient p. (The 
testing of p < py against p > py is treated in Problem 10.) The distribution 
of the sample correlation coefficient has not only monotone likelihood ratio 
as shown in Problem 10, but is in fact STP, [Karlin (1968, Section 3.4)]. 
Hence there exist tests of both H and H’ which are UMP among all tests 
that are both invariant and unbiased. 

Another case in which the combination of invariance and unbiasedness 
appears to offer a promising approach is the Behrens—Fisher problem. Let 
X,,..., Xm and Y,,..., Y, be samples from normal distributions N(£, o?) 
and N(n, 7?) respectively. The problem is that of testing H:n < & (or 
n = £) without assuming equality of the variances o? and r°. A set of 
sufficient statistics for (£, n, o, 7) is then (X, Y, S2, S?), where Sz = XX X, 
- X)?/(m — 1) and S; = X(Y, — Y)?/(n — 1). Adding the same constant 
to X and Y reduces the problem to Y — X, $7, S2, and multiplication of all 
variables by a common positive constant to (Y — X)/ (S + SẸ and S}/S¥. 
s would expect any reasonable invariant rejection region to be of the 
orm 


(15) iu ZI 


ys; Si s 


for some suitable function g. If this test is also to be unbiased, the 
probability of (15) must equal a when 7 = £ for all values of r/o. It has 
been shown by Linnik and others that only pathological functions g with 
this property can exist. [This work is reviewed by Pfanzagl (1974).] How- 
ever, approximate solutions are available which provide tests that are 
satisfactory for all practical purposes. These are the Welch approximate 
t-solution described in Chapter 5, Section 4, and the Welch-Aspin test. Both 
are discussed, and evaluated, in Scheffé (1970) and Wang (1971); see also 
Chernoff (1949), Wallace (1958), and Davenport and Webster (1975). 

The property of a test $; being UMP invariant is relative to a particular 
group G,, and does not exclude the possibility that there might exist another 
test $, which is UMP invariant with respect to a different group G;. Simple 
instances can be obtained from Examples 8 and 11. 


Example 8. (continued). If G, is the group G of Example 8, a UMP invariant 
test of H: 0 < 6, against 8 > @, rejects when Y, + --- +Y, > C. Let G, be the 
group obtained by interchanging the role of Z and Y,. Then a UMP invariant test 
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with respect to G, rejects when Z + Y, + --- + Y, > C. Analogous UMP invariant 
tests are obtained by interchanging the role of Z and any one of the other Y’s, and 
further examples by applying the transformations of G in Example 8 to more than 
one variable. In particular, if it is applied independently to all n + 1 variables, only 
the constants remain invariant, and the test ¢ = a is UMP invariant. 


Example 11.* For another example, let (X;, X12) and (X71, X22) be indepen- 
dent and have bivariate normal distributions with zero means and covariance 


matrices 
2 
9 P%% 
2 
pao 90 


Suppose that these matrices are nonsingular, or equivalently that |p| * 1, but that 
9,0, p, and A are otherwise unknown. The problem of testing A = 1 against 
A > 1 remains invariant under the group G, of all nonsingular transformations 


Ao] Apoo 
Apoo, Ao? ] 


Xh = bX, 
Yid xu DR RO 
Since the probability is 0 that Xj, X22 = X,? X, the 2 x 2 matrix ( X, ;) is nonsingu- 
lar with probability 1, and the sample space can therefore be restricted to be the set 
of all nonsingular such matrices. À maximal invariant under the subgroup corre- 
sponding to b = 1 is the pair (Xj, Xx). The argument of Example 6 then shows 
that there exists a UMP invariant test under G, which rejects when X3,/X;; > C. 
By interchanging 1 and 2 in the second subscript of the X’s one sees that under 
the corresponding group G; the UMP invariant test rejects when X},/ X) >C. 
A third group leaving the problem invariant is the smallest group containing both 
G, and G,, namely the group G of all common nonsingular transformations 
Xh = ai Xa + aX 


i , (i21). 

Xh = ay Xa + an Xn ( ) 
Given any two nonsingular sample points Z = (X;;) and Z’ = (Xj;), there exists a 
nonsingular linear transformation A such that Z’ = AZ. There are therefore no 
invariants under G, and the only invariant size-a test is $ = a. It follows vacuously 
that this is UMP invariant under G. 


7. ADMISSIBILITY 


Any UMP unbiased test has the important property of admissibility (Prob- 
lem 1 of Chapter 4), in the sense that there cannot exist another test which 
is uniformly at least as powerful and against some alternatives actually more 
Powerful than the given one. The corresponding property does not neces- 
sarily hold for UMP invariant tests, as is shown by the following example. 


*Due to Charles Stein. 
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Example 11. (continued). Under the assumptions of Example 11 it was seen that 
the UMP invariant test under G is the test p = a which has power B(A) = a. On 
the other hand, X,, and X; are independently distributed as N(0, o?) and 
N(0, Ao?). On the basis of these observations there exists a UMP test for testing 
A = 1 against A > 1 with rejection region X2,/X2, > C (Chapter 3 Problem 38). 
The power function of this test is strictly increasing in A and hence > a for all 
A>1. 


Admissibility of optimum invariant tests therefore cannot be taken for 
granted but must be established separately for each case. 

We shall distinguish two slightly different concepts of admissibility. A 
test pọ will be called a-admissible for testing H: 0 € Qy against a class of 
alternatives 0 € Q’ if for any other level-a test q 


(16) Eyp(X) > Ejg(X) forall 0E% 


implies Egp( X) = EPo(X) for all 0 € Q’. This definition takes no account 
of the relationship of E,p(X) and E,go(X) for 0 E Qy beyond the 
requirement that both tests are of level a. A concept closer to the 
decision-theoretic notion of admissibility discussed in Chapter 1, Section 8, 
defines o, to be d-admissible for testing H against Q’ if (16) and 


(7) . Eyp(X) < Ejp (X) forall 0€8Q, 


jointly imply Ej9( X) = Eygo( X) for all à € Qy U Q (see Problem 20). 

Any level-a test pọ that is a-admissible is also d-admissible provided no 
other test p exists with Eyp(X) = Eypo(X) for all 0 & Q^ but Ejg( X) + 
Eo@o(X) for some 0 € Qy. That the converse does not hold is shown by the 
following example. 


E Example 12. Let X be normally distributed with mean £ and known variance 
9^. For testing H: < —lor > 1 against Q: £ = 0, there exists a level-a test Po» 
which rejects when C, « X « C, and accepts otherwise, such that (Problem 21) 

E;po( X) < E. ;w(X)-a for £x-1 
and 


E po(X) < E(X) =a <a fo £241 


A slight modification of the proof of Theorem 6 of Chapter 3 shows that pọ is the 
unique test maximizing the power at = 0 subject to 


E,p(X) <a@ for£s 1 and E,9(X) <a’ for 21, 


and hence that g is d-admissible. 
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On the other hand, the test p with rejection region |X| < C, where Ej... ,9(X) 
= E,.,9(X) = a, is the unique test maximizing the power at = 0 subject to 
E,9(X) <a for < -10r > 1, and hence is more powerful against Q than po, 
so that q is not a-admissible. 


A test that is admissible under either definition against Q' is also 
admissible against any Q” containing Q’ and hence in particular against the 
class of all alternatives Qg =Q — Qy. The terms a- and d-admissible 
without qualification will be reserved for admissibility against Q,. Unless a 
UMP test exists, any a-admissible test will be admissible against some 
Q’ C Qy and inadmissible against others. Both the strength of an admissi- 
bility result and the method of proof will depend on the set w. 

Consider in particular the admissibility of a UMP unbiased test men- 
tioned at the beginning of the section. This does not rule out the existence of 
a test with greater power for all alternatives of practical importance and 
smaller power only for alternatives so close to H that the value of the power 
there is immaterial. In the present section, we shall discuss two methods for 
proving admissibility against various classes of alternatives. 


Theorem 8. Let X be distributed according to an exponential family with 
density 


n9) = cteo| Éan) 
x 


with respect to a a-finite measure over a Euclidean sample space (€, £), 
and let Q be the natural parameter space of this family. Let Qy and Q' be 
disjoint nonempty subsets of 9, and suppose that po is a test of H:8 EQ, 
based on T = (T, ..., T,) with acceptance region Ay which is a closed convex 
subset of R* possessing the following property: If Ao N (Eat; > c) is empty 
for some c, there exists a point 0* € Q and a sequence À, > œ such that 
0* + A,a € Q [where N, is a scalar and a = (ay... a,)]. Then if A is any 
other acceptance region for H satisfying 


P(XeA4)xP(XeA) forall 8€ o, 


A is contained in Ao, except for a subset of measure 0, i.e. u(4 N Âo) = 0. 


Proof. Suppose to the contrary that p(A n Ay) > 0. Then it follows 
from the closure and convexity of Ao that there exist a € R® and a real 


number c such that 


(18) A, (t: Laits > c} is empty 
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and 
(19) An {t: Lat, > c} has positive p-measure, 


that is, the set A protrudes in some direction from the convex set Ay. We 
shall show that this fact and the exponential nature of the densities imply 
that 


(20) P,(A) > P(A)  forsome 6€ 0’, 


which provides the required contradiction. Let pọ and denote the 
indicators of Ay and A respectively, so that (20) is equivalent to 


f Io) —(t)| dP(t)>0 forsome 0€ 9’. 


If 0 = 0* + X,a € Q, the left side becomes 


C(0* +X, 
Se [Io = e(t)] e^ Ean- ap, (t). 


Let this integral be I7 + J,, where I} and I; denote the contributions 
over the regions of integration (1:Ya,f; » c) and (t:Xa,t; € c) respec- 
tively. Since J; is bounded, it is enough to show that ^ > co as n > 00. 
By (18), p(t) = 1 and hence p(t) — p(t) > 0 when Xa,t, > c, and by (19) 


n{ g(t) — e(t) » 0. and Yat, > c) > 0. 


This shows that I7 — oo as À, — co and therefore completes the proof. 


Corollary 2. Under the assumptions of Theorem 8, the test with accep- 
tance region Ao is d-admissible. If its size is a and there exists a finite point 9% 
r the closure Qy of Qy for which Eg Po( X) = a, then po is also a-admissi- 

e. 


Proof. 


(i) Suppose ¢ satisfies (16). Then by Theorem 8, (x) < p(x) (a.e. 
L). If p(x) < p(x) on a set of positive measure, then Eyp(X) < 
Eq@(X) for all à and hence (17) cannot hold. 

(ii) By the argument of part (i), (16) implies a = Ey p(X) < Ea p X. 
and hence by the continuity of Egp(X) there exists a point @€ Qy 
for which a < E,p(X). Thus q is not a level-a test. 
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Theorem 8 and the corollary easily extend to the case where the competi- 
tors 9 of pọ are permitted to be randomized, but the assumption that pọ is 
nonrandomized is essential. Thus, the main applications of these results are 
to the case that p is absolutely continuous with respect to Lebesgue 
measure. The boundary of Ay will then typically have measure zero, so that 
the closure requirement for Ag can be dropped. 


Example 13. Normal mean. If X,,..., X, is a sample from the normal distribu- 
tion N(£, o?), the family of distributions is exponential with T, = X, T, = XX?, 
0, = n£/o?, 0, = —1/207. Consider first the one-sided problem H: 0, < 0, K: 6, 
> 0 with a < 1. Then the acceptance region of the t-test is 4: )/ /T; < C 
(C > 0), which is convex [Problem 22(i)]. The alternatives 0 € Q' C K will satisfy 
the conditions of Theorem 8 if for any half plane a,t, + azt, > c that does not 
intersect the set 4 < Cyt, there exists a ray (0f + Aa,, 07 + Aa;) in the direction 
of the vector (a, a) for which (Uf + Aa,, 0f + Aa,) € for all sufficiently large 
À. In the present case, this condition must hold for all a, > 0 > a;. Examples of 
sets Q satisfying this requirement (and against which the ‘test is therefore 
admissible) are 


£ 
91:0, > ky or zz > ki 


and 


Lihue 
Zika Ot = k- 


On the other hand, the condition is not satisfied for 0’: £ > k (Problem 22). 
Analogously, the acceptance region A : Tj < CT, of the two-sided t-test for 
testing H: 0, — 0 against 0, * 0 is convex, and the test is admissible against 


Q : £/o?| > k, and Q5 : |£/a| > kp. 
In decision theory, a quite general method for proving admissibility 
consists in exhibiting a procedure as a unique Bayes solution. In the present 
case, this is justified by the following result, which is closely related to 
Theorem 7 of Chapter 3. 
Theorem 9. ` Suppose the set (x: fo(x) > 0) is independent of 0, and let 
a o-field be defined over the parameter space St, containing both Qy and € 


and such that the densities fo(x) (with respect to p) of x are jointly 
measurable in 0 and x. Let Ag and A, be probability distributions over this 


o-field with A)(Qy,) = Ay(Qx) = 1, and let 


hj(x) = ffi) 4A,(8). 


310 INVARIANCE [6.7 
Suppose qx is a nonrandomized test of H against K defined by 


h(x 
odh x Hex, 


and that p{ x: h,(x)/ho(x) = k} = 0. 

(i) Then pọ is d-admissible for testing H against K. 

(ii) Let supg, E9?o(X) =a and w = (0: Eypo(X) = a}. If o C Ry 
and No(e) = 1, then y is also a-admissible. 

Gii) If A, assigns probability 1 to 2’ C Qx, the conclusions of (i) and (ii) 
apply with Q’ in place of Qg. 

Proof. (iy Suppose is any other test, satisfying (16) and (17) with 
Q = Qg. Then also 


JEX) 4h (0) < f Eyos (X) 4A (0) 


and 
[Ev (X) dA (0) = f Eyes (X) 4A,(8).- 


By the argument of Theorem 7 of Chapter 3, these inequalities are equiv- 
alent to 


Jo(x)ho(x) du(x) < foo(x)ho(x) du(x) 


and 


Jolla) du(x) > fev) G9) du(x), 


and the h,(x) (i = 0,1) are probability densities with respect to p. This 
contradicts the uniqueness of the most powerful test of họ against A; at 
level fepo(x)Ao(x) du(x). 

(ii): By assumption, /Egpo(x) dA.o(0) = a, so that gp is a level-a test of 
ho. If @ is any other level-a test of H satisfying (16) with 2’ = Qx, it is also 
a level-a test of A; and the argument of part (i) can be applied as before. 

(iii): This follows immediately from the proofs of (i) and (ii). 

Example 13. (continued). In the two-sided normal problem of Example 13 with 
H:&=0, K: £ * 0 consider the class 9, , of alternatives (£, o) satisfying 


1 bn 
atq © acq 


(21) o = 


=o << oo 
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for some fixed a, b > 0, and the subset w of Q, of points (0, 0?) with o? < 1/a. 
Let Ag, A, be distributions over w and Q/ , defined by the densities [Problem 
23()] 


Q 
Xo(n) = lasy” 
and 
C, e DE fam) 
EE 


Straightforward calculation then shows [Problem 23(ii)] that the densities hy and h, 
of Theorem 9 become 


and 


h(x) = 


so that the Bayes test pọ of Theorem 9 rejects when x?/Xx]» k and hence 
reduces to the two-sided t-test. , 

The condition of part (ii) of the theorem is clearly satisfied so that the t-test is 
both d- and a-admissible against €», ,. x F " 

When dealing with invariant tests, it is of particular interest to consider admissi- 
bility against invariant classes of alternatives. In the case of the two-sided test po, 
this means sets 2 depending only on l/a]. It was seen in Example 13 that qy is 
admissible against Q" :|£/o| > B for any B, that is, against distant alternatives, and 
it follows from the test being UMP unbiased or from Example 13 (continued) that 
Po is admissible against 9: |{/o| < A for any A > 0, that is, against alternatives 
close to H. This leaves open the question whether q is admissible against sets 
Q^:0 <A < |£/a| < B < co, which include neither nearby nor distant alternatives. 
It was in fact shown by Lehmann and Stein (1953) that gp is admissible for testing 
H against |£|/a — 8 for any ô> 0 and hence that it is admissible against any 
invariant Q. It was also shown there that the one-sided t-test of H: £ — 0 is 
admissible against £/o = 6’ for any 8’ > 0. These results will not be proved here. 
The proof is based on assigning to log o the uniform density on (-N, N) and 
letting N — oo, thereby approximating the “improper” prior distribution which 
assigns to log ø the uniform distribution on (— oo, oo), that is, Lebesgue measure. 

That the one-sided t-test p, of H:& < 0 is not admissible against all Q^ is 
shown by Brown and Sackrowitz (1984), who exhibit a test p satisfying 


E, 9 (X) < E, P(X) forall £«0, 0«o-« oo 
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and 
Ep P(X) > Ee oi(X) forall 0«& «£«£ «o, 0«o« o. 


Example 14. Normal variance. For testing the variance a? of a normal distribu- 
tion on the basis of a sample X,,..., X, from N(£, o°), the Bayes approach of 
Theorem 9 easily proves a-admissibility of the standard test against any location 
invariant set of alternatives Q’, that is, any set Q’ depending only on o*. Consider 
first the one-sided hypothesis H: c < o and the alternatives Q': o = o, for any 
o, > 0). Admissibility of the UMP invariant (and unbiased) rejection region L( X, — 
X)? > C follows immediately from Chapter 3, Section 9, where it was shown that 
this test is Bayes for a pair of prior distributions (Ay, A,): namely, A, assigning 
probability 1 to any point (¢,,0,), and A, putting o = a, and assigning to £ the 
normal distribution N(£,, (o? — a2)/n). Admissibility of E(X, — X)" < C when 
the hypothesis is H: o > o and Q' = {($, 0o): o = 9,), o, < og, is seen by inter- 
changing A, and A;, a, and o;. 

A similar approach proves a-admissibility of any size-a rejection region 


(22) Y (x, — X) «C, or. 2:C, 


for testing H:o — ay against Q':(o = 0,} U (o = 0} (o; < a < o;). On Qy, 
where the only variable is £, the distribution A, for £ can be taken as the normal 
distribution with an arbitrary mean £j and variance (o2 — o2 )/n. On €, let the 
conditional distribution of £ given ø = c; assign probability 1 to the value £,, and 
let the conditional distribution of £ given o = e; be N(£&, (o? — o? )/n). Finally, 
let A, assign probabilities p and 1 — p to a = o, and o = o, respectively. Then 
the rejection region satisfies (22), and any constants C, and C, for which the test 
has size a can be attained by proper choice of p [Problem 24(i)]. 


The results of Examples 13 and 14 can be used as the basis for proving 
admissibility results in many other situations involving normal distributions. 
The main new difficulty tends to be the presence of additional (nuisance) 
means. These can often be eliminated by use of the following lemma. 


Lemma 3. For any given o? and M? > c? there exists a distribution ^, 
such that 


H(z) = f 


Ges dA (E) 


is the normal density with mean zero and variance M?. 


Proof. Let 6 = {/a, and let 0 be normally distributed with zero mean 
and variance 7°. Then it is seen [Problem 24(ii)] that 


e 1 1 p 
V2r oyl +7? ox - 207(1 + 2». | 


6.7] ADMISSIBILITY 313 


The result now follows by letting 7? = (M?/o7) — 1, so that o?(1 + 7?) = 
M?. 


Example 15, Let X,,...,Xmi Y,,..., Y, be samples from N(£, o?) and N(1, 7?) 
respectively, and consider the problem of testing H : r/o = 1 against r/o = A > 1. 


(i Suppose first that £ =n — 0. If A, and A, assign probability 1 to the 
points (09, To = 95) and (0,, 7, = Ac;) respectively, the ratio h, /h; of Theorem 9 is 
proportional to 


died mee] 


and for suitable choice of critical value and e, < op, the rejection region of the 
Bayes test reduces to 


The values a? and of can then be chosen to give this test any preassigned size o. 

Gi) If £ and n are unknown, then X, Y, S} = (X, — X)’, S; = X(Y, - Y) 
are sufficient statistics, and S2 and S2 can be represented as Sy = Yjci Uj, 
S? = D"=}V?, with the Uj, V, independent normal with means 0 and variances o? 
and 7? respectively. 

To c and 7 assign the distributions A, and A, of part (i) and conditionally, 
given o and r, let £ and 1 be independently distributed according to Aga, Ao, over 
Q, and A,,,A,, over Qp, with these four conditional distributions determined 


from Lemma 3 in such a way that 


vm e (m/208XR-8 GAy, (£) = vn e7 nob? dos (£); 
/279, 9 Vro 


and analogously for v. This is possible by choosing the constant M 2 of Lemma 3 
greater than both o2 and oj. With this choice of priors, the contribution from X 
and y to the ratio h,/hy of Theorem 9 disappears, so that /j/h reduces to the 
expression for this ratio in part (i), with Ex? andX y replaced by L(x, — X)" and 
L(y, — y)? respectively. 

This approach applies quite generally in normal problems with nuisance 
means, provided the prior distribution of the variances c?, 1?,... assigns 
probability 1 to a bounded set, so that M 2 can be chosen to éxceed all 
possible values of these variances. 

Admissibility questions have been considered not only for tests but 
also for confidence sets. These will not be treated here (but see Chap- 
ter 9, Example 10); a convenient entry to the literature is Cohen and 
Strawderman (1973). For additional results, see Hooper (1982b) and Arnold 
(1984). 
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8. RANK TESTS 


One of the basic problems of statistics is the two-sample problem of testing 
the equality of two distributions. A typical example is the comparison of a 
treatment with a control, where the hypothesis of no treatment effect is 
tested against the alternatives of a beneficial effect. This was considered in 
Chapter 5 under the assumption of normality, and the appropriate test was 
seen to be based on Student's r. It was also shown that when approximate 
normality is suspected but the assumption cannot be trusted, one is led to 
replacing the /-test by its permutation analogue, which in turn can be 
approximated by the original t-test. 

We shall consider the same problem below without, at least for the 
moment, making any assumptions concerning even the approximate form of 
the underlying distributions, assuming only that they are continuous. The 
observations then consist of samples X,,..., X„ and Y,,..., Y, from two 
distributions with continuous cumulative distribution functions F and G, 
and the problem becomes that of testing the hypothesis 


H,:G=F. 


If the treatment effect is assumed to be additive, the alternatives are 
G(y) = F(y — A). We shall here consider the more general possibility that 
the size of the effect may depend on the value of y (so that A becomes a 
nonnegative function of y) and therefore test H, against the one-sided 
alternatives that the Y’s are stochastically larger than the X’s, 


K,:G(z) < F(z) forallz, and GF. 


An alternative experiment that can be performed to test the effect of a 
treatment consists of the comparison of N pairs of subjects, which have 
been matched so as to eliminate as far as possible any differences not due to 
the treatment. One member of each pair is chosen at random to receive the 
treatment while the other serves as control. If the normality assumption of 
Chapter 5, Section 12, is dropped and the pairs of subjects can be consid- 
ered to constitute a sample, the observations (X, Y)... (Xy, Yy) are a 
sample from a continuous bivariate distribution F. The hypothesis of no 
effect is then equivalent to the assumption that F is symmetric with respect 


to the line y = x: 
H,: F(x, y) = F(y, x). 


Another basic problem, which occurs in many different contexts, con- 
cerns the dependence or independence of two variables. In particular, if 
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(Xi Y), ---, (Xy, Yy) is a sample from a bivariate distribution F, one will 
be interested in the hypothesis 


Hy: F(x, y) = G,(x)G,(y) 


that X and Y are independent, which was considered for normal distribu- 
tions in Section 15 of Chapter 5. The alternatives of interest may, for 
example, be that X and Y are positively dependent. An alternative formula- 
tion results when x, instead of being random, can, be selected for the 
experiment. If the chosen values are x, < ::: < xy and F, denotes the 
distribution of Y given x,, the Y's are independently distributed with 
continuous cumulative distribution functions F,,..., Fy. The hypothesis of 
independence of Y from x becomes 


Hy: Fy = -* = Fy; 


while under the alternatives of positive regression dependence the variables 
Y, are stochastically increasing with i. 

In these and other similar problems, invariance reduces the data so 
completely that the actual values of the observations are discarded and only 
certain order relations between different groups of variables are retained. It 
is nevertheless possible on this basis to test the various hypotheses in 
question, and the resulting tests frequently are nearly as powerful as the 
standard normal tests. We shall now carry out this reduction for the four 
problems above. 

The two-sample problem of testing H, against K, remains invariant 
under the group G of all transformations 


xi = p(x), yf = ely) (i=1,...,m, j=l”) 


such that p is continuous and strictly increasing. This follows from the fact 
that these transformations preserve both the continuity of a distribution and 
the property of two variables being either identically distributed or one 
being stochastically larger than the other. As was seen (with a different 
notation) in Example 3, a maximal invariant under G is the set of ranks 


(Ri S?) = (Ri. Ros St SH) 


of X,,..., X,; Yjy--+» Y, in the combined sample. Since the distribution of 
(Ri,..., R^; St... SI) is symmetric in the first m and in the last n 
variables for all distributions F and G, a set of sufficient statistics for 
(R', S") is the set of the X-ranks and that of the Y-ranks without regard to 
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the subscripts of the X’s and Y’s. This can be represented by the ordered 
X-ranks and Y-ranks 

Ri <is0= Rio ands Sí «5, 
and therefore by one of these sets alone since each of them determines the 
other. Any invariant test is thus a rank test, that is, it depends only on the 
ranks of the observations, for example on (S,,..., S,). 

That almost invariant tests are equivalent to invariant ones in the present 
context was shown first by Bell (1964). A streamlined and generalized 
version of his approach is given by Berk and Bickel (1968) and Berk (1970), 
who also show that the conclusion of Theorem 6 remains valid in this case. 

To obtain a similar reduction for H,, it is convenient first to make the 
transformation Z, = Y, — X, W, = X, + Y,. The pairs of variables (Z;, W;) 
are then again a sample from a continuous bivariate distribution. Under the 
hypothesis this distribution is symmetric with respect to the w-axis, while 
under the alternatives the distribution is shifted in the direction of the 
positive z-axis. The problem is unchanged if all the w’s are subjected to the 
same transformation w/ = A(w,), where A is 1:1 and has at most a finite 
number of discontinuities, and (Z,,..., Zy) constitutes a maximal invariant 
under this group. [Cf. Problem 2(ii).] 

The Z’s are a sample from a continuous univariate distribution D, for 
which the hypothesis of symmetry with respect to the origin, 


Hy: D(z) + D(-z) 21  forallz, 


is to be tested against the alternatives that the distribution is shifted toward 
positive z-values. This problem is invariant under the group G of all 
transformations 


zi p(zj) 9) (i9 N) 


such that p is continuous, odd, and strictly increasing. If z;,..., z; « 0 « 
zy. Zj, where i < e «P, and ji < «e <j, let 51,..., 5; denote 
the ranks of z,,..., z, among the absolute values |z,|,..., |z|, and rý,- -> T 
the ranks of [z,|..., |z;,| among |z;,...,|zy| The transformations p 
preserve the sign of each observation, and hence in particular also the 
numbers m and n. Since p is a continuous, strictly increasing function of 
|z|, it leaves the order of the absolute values invariant and therefore the 
ranks r/ and sj. To see that the latter are maximal invariant, let (z,,.--» Zw) 
and (z{,..., zy) be two sets of points with m’ = m, n’ = n, and the same 7/ 
and 5j. There exists a continuous, strictly increasing function on the positive 
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real axis such that |z/| = p(|z;) and p(0) = 0. If p is defined for negative z 
by p(—z) = —p(z), it belongs to G and z; = p(z;) for all i, as was to be 
proved. As in the preceding problem, sufficiency permits the further reduc- 
tion to the ordered ranks r, < -*: «r, and s, < --- < s,. This retains 
the information for the rank of each absolute value whether it belongs to a 
positive or negative observation, but not with which positive or negative 
observation it is associated. 

The situation is very similar for the hypotheses H, and H,. The problem 
of testing for independence in a bivariate distribution against the alterna- 
tives of positive dependence is unchanged if the X, and Y, are subjected to 
transformations X/ = p( X;), Y/ = A(Y;) such that p and A are continuous 
and strictly increasing. This leaves as maximal invariant the ranks 
(Ri,..., RA) of (X,,..., Xy) among the X's and the ranks (S1,..., Sy) of 
(Y,,..., Yy) among the Y's. The distribution of (Ri, Si); -e (Ry Sx) is 
symmetric in these N pairs for all distributions of (X, Y). It follows that a 
sufficient statistic is (S,,..., Sy) where (1, S)... (N, Sy) is a permutation 
of (Ri, S{),...,(Riy, Sq) and where therefore S; is the rank of the variable 
Y associated with the ith smallest X. 

The hypothesis H, that Y;,..., Y, constitutes a sample is to be tested 
against the alternatives K, that the Y; are stochastically increasing with i. 
This problem is invariant under the group of transformations y/ = p(y;) 
where p is continuous and strictly increasing. A maximal invariant under 
this group is the set of ranks Sj,..., Sy of Ys... Yy- 

Some invariant tests of the hypotheses H, and H, will be considered in 
the next two sections. Corresponding results concerning H, and H, are 
given in Problems 46-48. 


9. THE TWO-SAMPLE PROBLEM 


The problem of testing the two-sample hypothesis H: G = F against the 
one-sided alternatives K that the Y’s are stochastically larger than the X's 
is reduced by the principle of invariance to the consideration of tests based 
on the ranks S, < --: < S, of the Y's. The specification of the S, is 
equivalent to specifying for each of the N =m + n positions within the 
combined sample (the smallest, the next smallest, etc.) whether it is occupied 
by an x or a y. Since for any set of observations n of the N positions are 
Occupied by y's and since the s ) possible assignments of n positions to 
the y's are all equally likely when G — F, the joint distribution of the S, 
under H is 


Q3) P(S 75... 8,78) = (A) 
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for each set 1 < s <s,< --: < s, < N. Any rank test of H of size 
m N 
a=k/ ( 2 ) 
therefore has a rejection region consisting of exactly k points (5;,..., Sn). 


For testing H against K there exists no UMP rank test, and hence no 
UMP invariant test. This follows for example from a consideration of two 
of the standard tests for this problem, since each is most powerful among all 
rank tests against some alternative. The two tests in question have rejection 
regions of the form 


(24) h(s,) * --- +h(s,) > C. 


One, the Wilcoxon two-sample test, is obtained from (24) by letting A(s) = s, 
so that it rejects H when the sum of the y-ranks is too large. We shall show 
below that for sufficiently small A, this is most powerful against the 
alternatives that F is the logistic distribution F(x) = 1/(1 + e^"), and that 
G(y) = F(y — A). The other test, the normal-scores test, has the rejection 
region (24) with A(s) = E(W,,)), where Wy) < --- < Wy, is an ordered 
sample of size N from a standard normal distribution.’ This is most 
powerful against the alternatives that F and G are normal distributions with 
common variance and means £ and 7 = £ + A, when A is sufficiently small. 

To prove that these tests have the stated properties it is necessary to 
know the distribution of (S,,..., S,) under the alternatives. If F and G 
have densities f and g such that f is positive whenever g is, the joint 
distribution of the S, is given by 


(VW) — 8(Yooo) (N). 


(25)..P(81 = 5,9. S m) E aie 
ja (Ye). (V) 


where Va) < ++- < Vy, is an ordered sample of size N from the distribu- 
tion F. (See Problem 29.) Consider in particular the translation (or shift) 
alternatives 


(y) - f(y - 4), 
and the problem of maximizing the power for small values of A. Suppose 


* Tables of the expected order statistics from a normal distribution are given in Biometrika 
Tables for Statisticians, Vol. 2, Cambridge U. P., 1972, Table 9. For additional references, see 
David (1981, Appendix, Section 3.2). 
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that f is differentiable and that the probability (25), which is now a 
function of A, can be differentiated with respect to A under the expectation 
sign. The derivative of (25) at A = 0 is then 


(Von) f (Vo) 


etes FORE C ACA 


ð 
34 Pat Ss = 5 Spz sh) 


Since under the hypothesis the probability of any ranking is given by (23), it 
follows from the Neyman-Pearson lemma in the extended form of Theorem 
5, Chapter 3, that the derivative of the power function at A — 0 is 
maximized by the rejection region 


(26 - E 
) E (Voo) 


The same test maximizes the power itself for sufficiently small A. To see this 


let s denote a general rank point (s,,...,5,), and denote by s the rank 
point giving the jth largest value to the left-hand side of (26). If 


a?) 


the power of the test is then 


k k 1 ô ; 
aS Di wae dai] — P, ( s? EE |p 
B(A) Eels?) Lr (%) FAJA als is 


Since there is only a finite number of points 5, there exists for each j a 
number A, > 0 such that the point s also gives the jth largest value to 
P(5) for all A < Aj. If A is less than the smallest of the numbers 


A; y=]; 


the test also maximizes B(A). 
If f(x) is the normal density N(£, 07), then 


, 


Yit PP s M e 
iri Goes ) 


o? 
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and the left-hand side of (26) becomes 


Lm a: G1 
G) 
= —2,E|W, 
2 o? " X ( ia) 
where Wa) < +++ < Wy is an ordered sample from N(0, 1). The test that 


maximizes the power against these alternatives (for sufficiently small A) is 
therefore the normal-scores test. 
In the case of the logistic distribution, 


$ er, 
F(x) = Teese f(x) = Gren)? 
and hence 
(x) | 
fx) Me 2F(x) Ed. 


The locally most powerful rank test therefore rejects when LE[F(V;,,))] > C. 
If V has the distribution F and 0 < y < 1, 


P(F(V) <y} =P{V< F-(y)) = F[F-(y)] = y, 


so that U = F(V) is uniformly distributed over (0, 1).* The rejection region 
can therefore be written as LE(U,,)) > C, where Uy < +++ < Uy) is an 
ordered sample of size N from the uniform distribution U(0, 1). Since 
E(U,,,) = s,/(N + 1), the test is seen to be the Wilcoxon test. 

Both the normal-scores test and the Wilcoxon test are unbiased against 
the one-sided alternatives K. In fact, let @ be the critical function of any 
test determined by (24) with h nondecreasing. Then is nondecreasing in 
the y’s, and the probability of rejection is a for all F = G. By Lemma 3 of 
Chapter 5 the test is therefore unbiased against all alternatives of K. 

It follows from the unbiasedness properties of these tests that the most 
powerful invariant tests in the two cases considered are also most powerful 
against their respective alternatives among all tests that are invariant and 
unbiased. The nonexistence of a UMP test is thus not relieved by restricting 
the tests to be unbiased as well as invariant. Nor does the application of the 
unbiasedness principle alone lead to a solution, as was seen in the discussion 
of permutation tests in Chapter 5, Section 11. With the failure of these two 


"This transformation, which takes a random variable with continuous distribution F into a 
uniformly distributed variable, is known as the probability integral transformation. 
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principles, both singly and in conjunction, the problem is left not only 
without a solution but even without a formulation. A possible formulation 
(stringency) will be discussed in Chapter 9. However, the determination of a 
most stringent test for the two-sample hypothesis is an open problem. 

Both tests mentioned above appear to be very satisfactory in practice. 
Even when F and G are normal with common variance, they are nearly as 
powerful as the t-test. To obtain a numerical comparison, suppose that the 
two samples are of equal size, and consider the ratio n*/n of the number of 
observations required by two tests to obtain the same power f against the 
same alternative. Let m=n and m* = n* = g(n) be the sample sizes 
required by one of the rank tests and the t-test respectively, and suppose (as 
is the case for the tests under consideration) that the ratio n*/n tends to a 
limit e independent of « and f as n > co. Then e is called the asymptotic 
efficiency of the rank test relative to the t-test. Thus, if in a particular case 
e — 1, then the rank test requires approximately twice as many observations 
as the t-test to achieve the same power. 

In the particular case of the Wilcoxon test, e turns out to be equal to 
3/m ~ 0.95 when F and G are normal distributions with equal variance. 
When F and G are not necessarily normal but differ only in location, e 
depends on the form of the distribution. It is always > 0.864, but may 
exceed 1 and can in fact be infinite! The situation is even more favorable 
for the normal-scores test. Its asymptotic efficiency relative to the t-test is 
always > 1 when F and G differ only in location; it is 1 in the particular 
case that F is normal (and only then). 

The above results do not depend on the assumption of equal sample 
sizes; they are also valid if m/n and m*/n* tend to a common limit p as 
n — oo where 0 <p < co. At least in the case that F is normal, the 
asymptotic results agree well with those found for very small samples. For a 
more detailed discussion of these and related efficiency results, see for 
example, Lehmann (1975), Randles and Wolfe (1979), and Blair and 
Higgins (1980). . 

It was seen in Chapter 5, Sections 4 and 11, that both the size and the 
power of the t-test and its permutation version are robust against nonnor- 
mality, that is, that the actual size and power, at least for large m and n, are 
approximately equal to the values asserted by the normal theory even when 
F is not normal. The two tests are thus performance-robust: under mild 
assumptions on F, their actual performance is, asymptotically, independent 
of F. However, as was pointed out in Chapter 5, Section 4, the insensitivity 
of the power to the shape of F is not as advantageous as may appear at first 
sight, since the optimality of the t-test is tied to the assumption of normal- 


* Upper bounds for certain classes of distributions are given by Loh (1984). 
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ity. The above results concerning the efficiency of the Wilcoxon and 
normal-scores tests show in fact that for many distributions F the t-test is 
far from optimal, so that the efficiency and optimality properties of t are 
quite nonrobust. 

The most ambitious goal in the nonparametric two-sample shift model 
(46) of Chapter 5 would be to find a test which asymptotically preserves the 
optimality for arbitrary F which the t-test possesses exactly in the normal 
case. Such a test should have asymptotic efficiency 1 not with respect to a 
fixed test, but for each possible true F with respect to the tests which are 
asymptotically most powerful for that F. Such adaptive tests (which achieve 
simultaneous optimality by adapting themselves to the unknown F) do in 
fact exist if F is sufficiently smooth, although they are not yet practical. 
Their possibility was first suggested by Stein (1956b), whose program has 
been implemented for point-estimation problems [see for example Beran 
(1974), Stone (1975), and Bickel (1982)], but not yet for testing problems. 

For testing H : G = F against the two-sided alternatives that the Y 's are 
either stochastically smaller or larger than the X's, two-sided versions of the 
rank tests of this section can be used. In particular, suppose that 4 is 
increasing and that A(s) + h(N + 1 — s) is independent of s, as is the case 
for the Wilcoxon and normal-scores statistics. Then under H, the statistic 
Xh(s;) is symmetrically distributed about nL" ,h(i)/N = p, and (24) sug- 
gests the rejection region 


1 n m 
|Xn(s) -u| = ru Y Als) —n E) 20 


The theory here is still less satisfactory than in the one-sided case. These 
tests need not even be unbiased [Sugiura (1965) and it is not known 
whether they are admissible within the class of all rank tests. On the other 
hand, the relative asymptotic efficiencies are the same as in the one-sided 
case. 

The two-sample hypothesis G — F can also be tested against the general 
alternatives G + F. This problem arises in deciding whether two products, 
two sets of data, or the like can be pooled when nothing is known about the 
underlying distributions. Since the alternatives are now unrestricted, the 
problem remains invariant under all transformations x/ = f(x;), y} = f») 
is 1,.. -,m, j =1,...,n, such that f has only a finite number of discon- 
tinuities. There are no invariants under this group, so that the only invariant 
test is @(x, y) = a. This is however not admissible, since there do exist tests 
of H that are strictly unbiased against all alternatives G + F (Problem 41). 
One of the tests most commonly employed for this problem is the Smirnov 
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test. Let the empirical distribution functions of the two samples be defined by 


a 
Si 20 Gods Jum mi d kz) = 


where a and b are the numbers of x's and y’s less or equal to z 
respectively. Then H is rejected according to this test when 


Accounts of the theory of this and related tests are given, for example, in 
Hajek and Sidák (1967), Durbin (1973), and Serfling (1980). 

Two-sample rank tests are distribution-free for testing H : G = F but not 
for the nonparametric Behrens-Fisher situation of testing H: 7 = & when 
the X's and Y’s are samples from F((x — £)/c) and F((y — 1)/7) with 
o,r unknown. A detailed study of the effect of the difference in scales on 
the levels of the Wilcoxon and normal-scores tests is provided by Pratt 
(1964). 


10. THE HYPOTHESIS OF SYMMETRY 


When the method of paired comparisons is used to test the hypothesis of no 
treatment effect, the problem was seen in Section 8 to reduce through 
invariance to that of testing the hypothesis 


Hi: D(z) + D(-z) = 1 forall z, 


which states that the distribution D of the differences Z, = Y, — X, (i = 
1,..., N) is symmetric with respect to the origin. The distribution D can be 
specified by the triple (p, F, G) where 


p=P{Z<0}, 'F(z)- P(IZIs z|Z < 0), 
G(z) = P(Z < z|Z > 0), 
and the hypothesis of symmetry with respect to the origin then becomes 
H:p-iG-F. 
Invariance and sufficiency were shown to reduce the data to the ranks 


S, < +++ < S, of the positive Z's among the absolute values |Z: Jn Zyl. 
The probability of S} = Sp- --» Sn = Sn is the probability of this event given 
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that there are n positive observations multiplied by the probability that the 
number of positive observations is n. Hence 


P{S, = 5,,...,5,=5,} 


rT (“Ja — p)'o" "Pr c(S, = 5... $,7 5,17} 


where the second factor is given by (25). Under H, this becomes 


1 
P{S, =5,,-.-, S, = 5) = 2N 
for each of the 
S/N 
= 9N 
x (x) a4 
n=0 
n-tuples (5,,...,5,) satisfying 1 < s, < --- «s, < N. Any rank test of 


size a = k/2" therefore has a rejection region containing exactly k such 
points (5,,..., Sp) 

The alternatives K of a beneficial treatment effect are characterized by 

_ the fact that the variable Z being sampled is stochastically larger than some 
random variable which is symmetrically distributed about 0. It is again 
suggestive to use rejection regions of the form h(s,) + --- +A(s,) > C, 
where however n is no longer a constant as it was in the two-sample 
problem, but depends on the observations. Two particular cases are the 
Wilcoxon one-sample test, which is obtained by putting A(s) — s, and the 
analogue of the normal-scores test with A(s) = E(W,,)) where Wy) < ++: 
< Wy) are the ordered values of |V,|,..., |Vy|, the V’s being a sample from 
N(0,1). The W’s are therefore an ordered sample of size N from a 
distribution with density /2/m e^" 7? for w > 0. 

As in the two-sample problem, it can be shown that each of these tests is 
most powerful (among all invariant tests) against certain alternatives, and 
that they are both unbiased against the class K. Their asymptotic efficien- 
cies relative to the t-test for testing that the mean of Z is zero have the same 
values 3/7 and 1 as the corresponding two-sample tests, when the distribu- 
tion of Z is normal. 

In certain applications, for example when the various comparisons are 
made under different experimental conditions or by different methods, it 
may be unrealistic to assume that the variables Z,,..., Zy have a common 
distribution. Suppose instead that the Z, are still independently distributed 
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but with arbitrary continuous distributions D,. The hypothesis to be tested 
is that each of these distributions is symmetric with respect to the origin. 

This problem remains invariant under all transformations z/ = f;(z;) 
i= 1,..., N, such that each f, is continuous, odd, and strictly increasing. A 
maximal invariant is then the number n of positive observations, and it 
follows from Example 8 that there exists a UMP invariant test, the sign test, 
which rejects when n is too large. This test reflects the fact that the 
magnitude of the observations or of their absolute values can be explained 
entirely in terms of the spread of the distributions D,, so that only the signs 
of the Z’s are relevant. 

Frequently, it seems reasonable to assume that the Z’s are identically 
distributed, but the assumption cannot be trusted. One would then prefer to 
use the information provided by the ranks s, but require a test which 
controls the probability of false rejection even when the assumption fails. As 
is shown by the following lemma, this requirement is in fact satisfied for 
every (symmetric) rank test. Actually, the lemma will not require even the 
independence of the Z’s; it will show that any symmetric rank test 
continues to correspond to the stated level of significance provided only the 
treatment is assigned at random within each pair. 


Lemma 4. Let $(z;,-.-,Zy) be symmetric in its N variables and such 
that 


(27) Ej (Z,...., Zy) = a 


when the Z's are a sample from any continuous distribution D which is 
symmetric with respect to the origin. Then 


Q8) E6(Z,,..., Zy) - a 


if the joint distribution of the Z's is unchanged under the 2" transformations 
Z(- £Z, Zy md. 


Proof. The condition (27) implies 
ltzen tz.) 


Ji = 
(29) X eae ee zig! ae, 
Qi in) 
where the outer summation extends over all N! permutations (jis . . Jy) 


and the inner one over all 2% possible choices of the signs + and —. This is 
proved exactly as was Theorem 6 of Chapter 5. If in addition $ is 
Symmetric, (29) implies 


(30) AE tn 
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Suppose that the distribution of the Z’s is invariant under the 2” transfor- 
mations in question. Then the conditional probability of any sign combina- 
tion of Z,,..., Zy given |Z,|,-.-,|Zyl is 1/2^. Hence (30) is equivalent to 


(31) E[o(Z,,---,ZwIZib---|Zvl] =e a.e. 


and this implies (28) which was to be proved. 

The tests discussed above can be used to test symmetry about any known 
value 6, by applying them to the variables Z; — @. The more difficult 
problem of testing for symmetry about an unknown point @ will not be 
considered here. Tests of this hypothesis are discussed, among others, by 
Antille, Kersting, and Zucchini (1982), Bhattacharya, Gastwirth, and Wright 
(1982), Boos (1982), and Koziol (1983). 

As was pointed out in Section 5 of Chapter 5, the one-sample t-test is 
not robust against dependence. Unfortunately, this is also true—although 
to a somewhat lesser extent—of the sign and one-sample Wilcoxon tests 
[Gastwirth and Rubin (1971)]. 


11. EQUIVARIANT CONFIDENCE SETS 


Confidence sets for a parameter 0 in the presence of nuisance parameters 9 
were discussed in Chapter 5 (Sections 6 and 7) under the assumption that 0 
is real-valued. The correspondence between acceptance regions A4(06,) of the 
hypotheses H(60,) : 0 = 6, and confidence sets S(x) for @ given by (34) and 
(35) of Chapter 5 is, however, independent of this assumption; it is valid 
regardless of whether 0 is real-valued, vector-valued, or possibly a label for 
a completely unknown distribution function (in the latter case, confidence 
intervals become confidence bands for the distribution function). This 
correspondence, which can be summarized by the relationship 


(32) 8€S(x) ifandonlyif x € A(0), 


was the basis for deriving uniformly most accurate and uniformly most 
accurate unbiased confidence sets. In the present section, it will be used to 
obtain uniformly most accurate equivariant confidence sets. 

We begin by defining equivariance for confidence sets. Let G be a group 
of transformations of the variable X preserving the family of distributions 
( P. 9, (0, 8) € Q} and let G be the induced group of transformations of Q. 
If g(0, 9) = (6’, 9^), we shall suppose that 8’ depends only on g and 0 and 
not on #, so that g induces a transformation in the space of 0. In order to 
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keep the notation from becoming unnecessarily complex, it will then be 
cofivenient to write also 6’ = g@. For each transformation g € G, denote by 
g* the transformation acting on sets S in Ó-space and defined by 


(33) g*S = (g0:0€5), 


so that g*S is the set. obtained by applying the transformation g to each 
point @ of S. The invariance argument of Chapter 1, Section 5, then 
suggests restricting consideration to confidence sets satisfying 


(34) g*S(x)=S(gx) forall xe2%, gee. 


We shall say that such confidence sets are equivariant under G. This 
terminology avoids the impression created by the term invariance (used by 
some authors and in the first edition of this book) that the confidence sets 
remain unchanged under the transformation X’ = gX. If the transformation 
g is interpreted as a change of coordinates, (34) means that the confidence 
statement does not depend on the coordinate system used to express the 
data. The statement that the transformed parameter gô lies in S(gx) is 
equivalent to stating that 8 € g*~'S(gx), which is equivalent to the original 
statement 0 € S(x) provided (34) holds. 


Example 16. Let X,Y be independently normally distributed with means & " 
and unit variance, and let G be the group of all rigid motions of the plane, which is 
generated by all translations and orthogonal transformations. Here g — g for all 
8 € G. An example of an equivariant class of confidence sets is given by 


S(x,y) = ((G,0):(x-8 Gi - m $C}, 


the class of circles with radius VC and center (x, y). The set g*S(x, y) is the set of 
all points g(£,) with (£, y) € S(x,y), and hence is obje oy subjecting 
S(x, y) to the rigid motion g. The result is the circle with radius ¥C and center 


g(x, y), and (34) is therefore satisfied. 
In accordance with the definitions given in Chapters 3 and 5, a class of 


confidence sets for 6 will be said to be uniformly most accurate equivariant 
at confidence level 1 — a if among all equivariant classes of sets S(x) at 


that level it minimizes the probability 
P,,(0 S(X)} forall 0 +0. 


In order to derive confidence sets with this property from families of UMP 
invariant tests, we shall now investigate the relationship between equi- 
variance of confidence sets and invariance of the associated tests. 
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Suppose that for each 6, there exists a group of transformations Gy, 
which leaves invariant the problem of testing H(0,) : 0 = 65, and denote by 
G the group of transformations generated by the totality of groups Gy. 


Lemma 5. 

G) Let S(x) be any class of confidence sets that is equivariant under G, 
and let A(0) = (x:0,& S(x)); then the acceptance region A(@) is invariant 
under Gy for each 0. 

(ii) ‘If in addition, for each 6, the acceptance region A(05) is UMP 
invariant for testing H(0,) at level a, the class of confidence sets S(x) is 
uniformly most accurate among all equivariant confidence sets at confidence 
level 1 — a. 


Proof. (i): Consider any fixed 0, and let g € G,. Then 
gA(0) = (gx:6 © S(x)) = (x:6e S(g-ix)) = (x: 0 e g*-'s(x)) 
= (x:g8 € S(x)) = {x:0 € s(x)) = A(0). 


Here the third equality holds because S(x) is equivariant, and the fifth one 
because g € Gg and therefore gô = 6. 

(ii): If S'(x) is any other equivariant class of confidence sets at the 
prescribed level, the associated acceptance regions A’(@) by (i) define 
invariant tests of the hypotheses H(@). It follows that these tests are 


uniformly at most as powerful as those with acceptance regions A(@) and 
hence that 


Py 9{ 8’ E S(X)} «P, (€ eS(X)) forall 0° #9, 


as was to be proved. 

It is an immediate consequence of the lemma that if UMP invariant 
acceptance regions A(@) have been found for each hypothesis H(@) (in- 
variant with respect to G,), and if the confidence sets S(x) = (0: x € A(9)} 
are equivariant under G, then they are uniformly most accurate equivariant. 

Example 17. Under the assumptions of Example 16, the problem of testing 


£ = £o, 1 = mo is invariant under the group G;, ,, of orthogonal transformations 
about the point (£o, 10): nit 


X’ — fo = ag(X — $o) + ay2(¥ — 10), 
Y — 9 = ag(X — &) + as(Y — m); 


where the matrix (a; j) is orthogonal. There exists under this group a UMP invariant 
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test, which has the acceptance region (Problem 8 of Chapter 7) 


(X= &y + (Y- m) «C. 


Let Go be the smallest group containing the groups G; , for all £, y. Since this is a 
subgroup of the group G of Example 16 (the two groups actually coincide, but this 
is immaterial for the argument), the confidence sets (X — £)? + (Y — 9)? < C are 
equivariant under Gy and hence uniformly most accurate equivariant. 


Example 18. Let X,,..., X, be independently normally distributed with mean 
£ and variance o?. Confidence intervals for £ are based on the hypotheses H(£,) : £ 
= £o, which are invariant under the groups G,, of transformations X/ = a( X, — £o) 
+¢,(a * 0). The UMP invariant test of H()) has acceptance region 


y(n Fx 1)n|X £l «cC 
yxO - X) 


and the associated confidence intervals are 


ae c — - c — 
(35) X CR CRT ee ser b x 


The group G in the present case consists of all transformations g: X; = aX, + b 
(a + 0), which on £ induces the transformation g: £' = a£ + b. Application of the 
associated transformation g* to the interval (35) takes it into the set of points 
a£ + b for which £ satisfies (35), that is, into the interval with end points 


oF +b- cres RD REDE ok +b+ EE EO -3Y 


Since this coincides with the interval obtained by replacing X, in (35) with aX, + b, 
the confidence intervals (35) are equivariant under Go and hence uniformly most 
accurate equivariant. 


Example 19. In the two-sample problem of Section 9, assume the shift model in 
Which the X’s and Y's have densities f(x) and g(y) = f(y — A) respectively, and 
consider the problem of obtaining confidence intervals for the shift parameter A 
which are distribution-free in the sense that the coverage probability is independent 
of the true f. The hypothesis H(Aọ) : A = Ao can be tested, for example, by means 
of the Wilcoxon test applied to the observations X,, Y, — Ao, and confidence sets for 
A can then be obtained by the usual inversion process. The resulting confidence 
intervals are of the form Dj) < A < Demn+1-k) where Day < ::: < Dimny ate the 
mn ordered differences Y - X. [For details see Problem 39 and for fuller accounts 
nonparametric books such as Lehmann (1975) and Randles and Wolfe (1979).] By 
their construction, these intervals have a coverage probability 1-a which is 
independent of f. However, the invariance considerations of Sections 8 and 9 do not 
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apply. The hypothesis H(Aq) is invariant under the transformations X/ = p( X;), 
Y/ = p(Y, — Ao) + A, with p continuous and strictly increasing, but the shift 
model, and hence the problem under consideration, is not invariant under these 
transformations. 


12. AVERAGE SMALLEST EQUIVARIANT 
CONFIDENCE SETS 


In the examples considered so far, the invariance and equivariance proper- 
ties of the confidence sets corresponded to invariant properties of the 
associated tests. In the following examples this is no longer the case. 


Example 20. Let X,,...,X, be a sample from N(£, o°), and consider the 
problem of estimating 0°. 

The model is invariant under translations X/ = X, + a, and sufficiency and 
invariance reduce the data to S? = E(X, — X)?. The problem of estimating o^ by 
confidence sets also remains invariant under scale changes X/ = bX, S’ = bS, 
a’ = bo (0 < b), although these do not leave the corresponding problem of testing 
the hypothesis ø = a, invariant. (Instead, they leave invariant the family of these 
testing problems, in the sense that they transform one such hypothesis into another.) 
The totality of equivariant confidence sets based on S is given by 


2 
o 
(36) yet 


where A is any fixed set on the line satisfying 
1 
(37) ps ea) -1-a. 


That any set c^ € $^. A is equivariant is obvious. Conversely, suppose that 
oe es ) is an equivariant family of confidence sets for o?. Then C(S?) must 
satisfy b^C(S?) = C(b^S?) and hence 


2 


p m T 
c? € C(S?) if and only if si $g$€65)- cQ). 


which establishes (36) with A = C(1). 

Among the confidence sets (36) with A satisfying (37) there does not exist one 
that uniformly minimizes the probability of covering false values (Problem 55). 
Consider instead the problem of determining the confidence sets that are physically 
smallest in the sense of having minimum Lebesgue measure. This requires mini- 


mizing f, dv subject to (37). It follows from the Neyman-Pearson lemma that the 
minimizing A* is 


(38) A* = (v: p(v) > C), 
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where p(v) is the density of V = 1/S? when c = 1, and where C is determined by 
(37). Since p(v) is unimodal (Problem 56), these smallest confidence sets are 
intervals, aS? < o? < bS?. Values of a and b are tabled by Tate and Klett (1959), 
who also table the corresponding (different) values a', b' for the uniformly most 
accurate unbiased confidence intervals a'S < o? < b'S? (given in Example 5 of 
Chapter 5). 

Instead of minimizing the Lebesgue measure f, dv of the confidence sets A, one 
may prefer to minimize the scale-invariant measure 


39 b 
(39) [ = a 


To an interval (a, b), (39) assigns, in place of its length b — a, its logarithmic length 
log b — log a = log(b/a). The optimum solution A** with respect to this new 
measure is again obtained by applying the Neyman-Pearson lemma, and is given by 


(40) A** = {v: up(v) > C), 


which coincides with the uniformly most accurate unbiased confidence sets [Problem 
57(i)]. 

One advantage of minimizing (39) instead of Lebesgue measure is that it then 
does not matter whether one estimates o or a? (or o^ for some other power of r), 
since under (39), if (a, b) is the best interval for o, then (a’, b^) is the best interval 
for o" [Problem 57(ii)]. 


Example 21. Let X, (i — 1,...,r) be independently normally distributed as 
N(£,,1). A slight generalization of Example 17 shows that uniformly most accurate 
equivariant confidence sets for (£,,..., £,) exist with respect to the group G of all 
rigid transformations and are given by 


(41) yy by sc. 


Suppose that the context of the problem does not possess the symmetry which 
would justify invoking invariance with respect to G, but does allow the weaker 
assumption of invariance under the group Gp of translations Xj = X, + a,. The 
totality of equivariant confidence sets with respect to Gy is given by 


(42) (% = &)---) % = &) EA, 
where A is any fixed set in r-space satisfying 


(43) Fee ele Mita oe Vii Ck 


Since uniformly most accurate equivariant confidence sets do not exist (Problem 
55), let us consider instead the problem of determining the confidence sets of 
smallest Lebesgue measure. (This measure is invariant under Go.) This is given by 
(38) with v = (v,,..., v,) and p(v) the density of (%,---» X,) when ġ = +++ =$, 
= 0, and hence coincides with (41). 
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Example 22. In the preceding example, suppose that the X, are distributed as 
N(£,, o?) with o? unknown, and that a variable S? is available for estimating o. 
Of S? assume that it is independent of the X’s and that S?/o? has a x*-distribu- 
tion with f degrees of freedom. 

The estimation of (£,,...,£,) by confidence sets on the basis of X’s and S? 
remains invariant under the group Gy of transformations 


X! = bX, + a;, S'-bS, $ =b; +a; o’ = bo, 


and the most general equivariant confidence set is of the form 


X-& X= f, 
44 TRO MM , 
(44) (S55... 5.) e, 
where A is any fixed set in r-space satisfying 
X X, 
45 P, Teu -1-a. 
(45) vel =) e4] L-a 


The confidence sets (44) can be written as 
(46) eias 6r) © (X,,--. X,) — SA, 
where — SA is the set obtained by multiplying each point of A by the scalar — S. 
To see (46), suppose that C(X,,..., X,; S) is an equivariant confidence set for 
(&),...,§,). Then the r-dimensional set C must satisfy 
C(bX, + a,,..., bX, + aj; bS) = b[C(X,,.-.,.X,5 )] + (a... 2) 


for all a,,..., a, and all b > 0. It follows that (£,,..., £,) € C if and only if 


ee KE SAP OV TRY 
zé she! ie X) CO KiS) L eo) =A. 


ie s 


The equivariant confidence sets of smallest volume are obtained by choosing for 4 

the set A* given by (38) with v = (v,,...,v,) and p(v) the joint density of 

M/S,- X,/S) when £j = --- = £, = 0. This density is a decreasing function 

a Xv; (Problem 58), and the smallest equivariant confidence sets are therefore given 
y 


(47) X(X-&yscs. 


[Under the larger group G generated by all rigid transformations of (,,..., X.) 
together with the scale changes X’ = bX,, S’ = bS, the same sets have the stronger 
property of being uniformly most accurate equivariant; see Problem 59.] 


Examples 20-22 have the common feature that the equivariant con- 
fidence sets S(X) for 0 = (0,,...,6,) are characterized by an r-valued 
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pivotal quantity, that is, a function h(X, 8) = (A,(X, 0),..., h,(X, 0)) of 
the observations X and parameters @ being estimated that has a fixed 
distribution, and such that the most general equivariant confidence sets are 
of the form 


(48) h(X,0) € A 


for some fixed set A.* When the functions h, are linear in 0, the confidence 
sets C( X) obtained by solving (48) for @ are linear transforms of A (with 
random coefficients), so that the volume or invariant measure of C( X) is 
minimized by minimizing 


(49) [p(o m dv, ... do, 


for the appropriate p. The problem thus reduces to that of minimizing (49) 
subject to 


(50)  P,(h(X,6)€4) = fpu) dv,...dv,=1- a, 


where p(v,,...,0,) is the density of the pivotal quantity A(X, 0). The 
minimizing A is given by 


P(04,-++> 9) 

51 At = (pi ———— = C), 
p oes 
with C determined by (50). i 

The following is one more illustration of this approach. 


2 
Example 23. Let X,,..., Xm and Y, sd a samples from N(£,o ) and 
N(n, 7?) respectively, and consider the problem of estimating A = 77/6", Sufficiency 
and invariance under translations X/ = X; + 41, amy + a reduce the data. to 
S} = X(X, — X)? and S} = X(Y, — Y)". The problem of estimating A also remains 
invariant under the scale changes 


Xi -bX, Y) = bY, 0 < by, b; < co, 
which induce the transformations 
(52) St = bi Spod Spo bSy, | a= bo, fbr. 


*More general results concerning the relationship of equivariant confidence sets and pivotal 


quantities are given in Problems 78-81. 
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The totality of equivariant confidence sets for A is given by A/V € A, where 
V = S2/S5 and A is any fixed set on the line satisfying 


(53) nae) 717 


To see this, suppose that C(Sy, Sy) are any equivariant confidence sets for A. 
Then C must satisfy 


b 
(54) C( bySx, by) = 75 Cl Sys Sy), 
1 


and hence A € C(S,, Sy) if and only if the pivotal quantity V/A satisfies 


D SAS 
ET exci Sp) = C(1,1) =A. 


As in Example 20, one may now wish to choose A so as to minimize either its 
Lebesgue measure {, dv or the invariant measure /,(1/v) dv. The resulting con- 
fidence sets are of the form 


(55) p(v)» C and up(v)>C 


respectively. In both cases, they are intervals V/b < A < V/a [Problem 60(i)]. The 
values of a and b minimizing Lebesgue measure are tabled by Levy and Narula 
(1974); those for the invariant measure coincide with the uniformly most accurate 
unbiased intervals [Problem 60(ii)]. 


13. CONFIDENCE BANDS FOR A DISTRIBUTION 
FUNCTION 


Suppose that X = (X,,..., X,) is a sample from an unknown continuous 
cumulative distribution function F, and that lower and upper bounds Ly 
and My are to be determined such that with preassigned probability 1 — a 


the inequalities 
Ly(u) < F(u) < My(u) forall u 


hold for all continuous cumulative distribution functions F. This problem is 
invariant under the group G of transformations 


AA) | d m1, n, 


where 8 is any continuous strictly increasing function. The induced trans- 
formation in the parameter space is gF = F( g 
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If S(x) is the set of continuous cumulative distribution functions 
S(x) = (F: L,(u) < F(u) < M,(u) for allu}, 
then 
g*S(x) = (gF: L,(u) < F(u) < M,(u) for all u } 
= {F:L,[g—(u)] s F(u) < M, [g^ (u)] for all u}. 


For an equivariant procedure, this must coincide with the set 


The condition of equivariance is therefore 
Lx) "eom ax lg(u)] =L,(u), My) lis ENERO = M,(u) 
for all x and u. 
To characterize the totality of equivariant procedures, consider the 
empirical distribution function (EDF) T, given by 


i " 
T,(u) == for xa Su < Xay 170, n, 


where xq, < *** < xq is the ordered sample and where xo = — 00, 
X(n41) = oo. Then a necessary and sufficient condition for L and M to 
satisfy the above equivariance condition is the existence of numbers 
49,..., dy; df... , d, such that 


L,(u) =a,;, M,(u) =a} for Xa <4 < Xur 


That this condition is sufficient is immediate. To see that it is also necessary, 
let u, u’ be any two points satisfying xj) < u <u’ < Xr: Given any 
Jy --., y, and v with y € v € Yus1» there exist g, g’ € G such that 


gx) = £O) = Xi) g(v) =4, g'(v) =u’. 


If L., M, are equivariant, it then follows that L,(u') = L,(v) and L,(u) = 
L,(v), and hence that L,(u’) = L,(u) and similarly M,(u’) = M,(u), as 
was to be proved. This characterization shows L, and M, to be step 
functions whose discontinuity points are restricted to those of T,- 
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Since any two continuous strictly increasing cumulative distribution 
functions can be transformed into one another through a transformation g, 
it follows that all these distributions have the same probability of being 
covered by an equivariant confidence band. (See Problem 66.) Suppose now 
that F is continuous but no longer strictly increasing. If J is any interval of 
constancy of F, there are no observations in J, so that J is also an interval 
of constancy of the sample cumulative distribution function. It follows that 
the probability of the confidence band covering F is not affected by the 
presence of J and hence is the same for all continuous cumulative distribu- 
tion functions F. 

For any numbers a;, a; let A;, X, be determined by 


i i 
a;=—-A,, aj.=— + Wy. 
n n 


Then it was seen above that any numbers A,,..., 4,; A,,..., A, define a 
confidence band for F, which is equivariant and hence has constant prob- 
ability of covering the true F. From these confidence bands a test can be 
obtained of the hypothesis of goodness of fit F = F, that the unknown F 
equals a hypothetical distribution F}. The hypothesis is accepted if Fo lies 
entirely within the band, that is, if 


=A; < F(u) - T,(u) < N, 
forall xa <u< x; andal i=1,...,n. 


Within this class of tests there exists no UMP member, and the most 
common choice of the A’s is A; = A’, = A for all i. The acceptance region of 
the resulting Kolmogorov test can be written as 


(56) sp |F(u) - T,(u)| <A. 


—o«u 


Tables of the null distribution of the Kolmogorov statistic are given by 
Birnbaum (1952). For large n, approximate critical values can be obtained 
from the limit distribution K of yn sup |Fy(u) — T,(u)| due to Kolmogorov 
and tabled by Smirnov (1948). Derivations of K can be found, for example, 
in Feller (1948), Hajek and Sidák (1967), and Billingsley (1968). 

. Alternative goodness-of-fit tests are based on other measures of the 
distance between the cumulative distribution functions F and T,. Surveys 
dealing with properties of such tests, including tests for goodness of fit when 
the hypothesis specifies a parametric family rather than a single distribution, 
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are provided by Durbin (1973), Kendall and Stuart (1979, Chapter 30), 
Neuhaus (1979), and Tallis (1983). 


14. PROBLEMS 


Section 1 


l. Let G be a group of measurable transformations of (X, £) leaving 2 = { Py, 
6 € Q} invariant, and let T(x) be a measurable transformation to (7, 4). 
Suppose that T(x;) = T(x) implies T(gx,) = T(gx;) for all g € G, so that 
G induces a group G* on J through g*T(x) = T(gx), and suppose further 
that the induced transformations g* are measurable 2. Then G* leaves the 
family 27 = ( P7, 6 € Q} of distributions of T invariant. 


Section 2 


2. (i) Let Z be the totality of points x = (x,..., Xn) for which all coordinates 
are different from zero, and let G be the group of transformations 


xf = cx, ¢ > 0. Then a maximal invariant under G is 
(SQN x, , X1/ Xn- --» X,-1/X,) Where sgn x is 1 or —1 as x is positive or 
negative. 


Gi) Let 4 be the space of points x = (3;,.... Xy) for which all coordinates 
are distinct, and let G be the group of all transformations x; = f(x), 
i=1,...,n, such that f isa 1:1 transformation of the real line onto 
itself with at most a finite number of discontinuities. Then G is transitive 


over X. 
[(ii): Let x = (xj... Xn) and x’ = (xf... Xp) be any two points of Z. Let 
I\,..., J, be a set of mutually exclusive open intervals which (together with 


their end points) cover the real line and such that x, € Ij. Let If... Ij bea 
corresponding set of intervals for x(,..., Xn- Then there exists a transforma- 
tion f which maps each 7, continuously onto //, maps x, into Xj and maps 
the set of n — 1 end points of /,,..../, onto the set of end points of 


Hoste 


3. (i A sufficient condition for (8) to hold is that D is a normal subgroup of G. 
Gi) If G is the group of transformations x’ = ax + b, a # 0,-0 < b< co, 
then the subgroup of translations x’ = x + b is normal but the subgroup 
x’ = ax is not. 
[The defining property of a normal subgroup is that given d € D, g € G, there 
exists d’ € D such that gd = d'g. The equality s(x,) = $(X2) implies xX, = dx, 
for some d € D, and hence ex, = edx; = d'ex,. The result (i) now follows, 
since s is invariant under D.] 
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Section 3 


4. Let X,Y have the joint probability density f(x, y). Then the integral h(z) = 
[8 sf (y — z, y) dy is finite for almost all z, and is the probability density of 


5: 


Z= 


rax 


[Since P{Z < b) = f^ „h(z) dz, it is finite and hence h is finite almost 
everywhere.] 


(i) Let X = (X,..., X,) have probability density (/0")f[(x1 — 


Gi) 


£)/0,...,(x, — £)/0], where -œ < < œ, 0< 0 are unknown, and 
where f is even. The problem of testing f = f, against f = f, remains 
invariant under the transformations x/ = ax, + b (i — l,..., n), a * 0, 
—oo < b < oo, and the most powerful invariant test is given by the 
rejection region 


oo oo 
ji d v"? f (vx, + u,..., 0x, + u) dodu 
—o"*0 


oo 'o0 
> ef f u"? (ox, + u,..., 0x, + u) dedu. 


Let X =(X,,..., X,) have probability density f (x, — X avi; Xn 
= Dkw) where k < n, the w’s are given constants, the matrix (w;;) 
is of rank k, the B’s are unknown, and we wish to test f = fọ against 
f = fı: The problem remains invariant under the transformations x = x; 
+ Zh Gwgy, — 0 < y... Y, < 00, and the most powerful invariant 
test is given by the rejection region 


fev fal -E wbo Xn — E wb) dBy,..., dB, 
fff n -YXwmjB,.... x, - 12w8) dB,,.--, 4B, 


>C. 


[A maximal invariant is given by y = 


n n n 
s- Dua rd Pix ips ado ndspXssle e Xp tao TATA 


r=n-k+1 r=n-k+1 r=n-k+1 


for suitably chosen constants a,,.] 


6. Let X,,..., X,; Yis... Y, be samples from exponential distributions with 


G) 


densities a^ le^ 7" for x > £, and 7^ le- 0-7)" for y >n. 


For testing t/o < A against r/o > A, there exists a UMP invariant test 
with respect to the group G: X/ = aX, + b, Y' = aY, + c, a > 0, —% 
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< b, c < œ, and its rejection region is 


Y[ = min... »)] 2 
Y [x - min(,....x4)] ' 


(i) This test is also UMP unbiased. 
(iii) Extend these results to the case that only the r smallest X's and the s 
smallest Y's are observed. 


[(ii): See Problem 12 of Chapter 5.] 


If X,,..., X, and Y,,..., Y, are samples from N(£, o^) and N(n, 7?) respec- 
tively, the problem of testing ?-— o? against the two-sided alternatives 
72 + o? remains invariant under the group G generated by the transformations 
X! = aX, * b, Y/ = aY,+ c, a + 0, and X/ = Y, Y/ = X, There exists a 
UMP invariant test under G with rejection region 


Ec-Y* Eq-3., 
L(y =X) DH] 


[The ratio of the probability densities of W for r/o” = A and 1/0? =1is 
proportional to [(1 + w)/(A + w)]""? + [(1 + w)/ü + Aw)]|"^! for w z 1. 
The derivative of this expression is > 0 for all A.] 


W = max 


Section 4 
G) When testing H: p < po against K: p > Po by means of the test corre- 
sponding to (11), determine the sample size required to obtain power B 
against p = pj, a = .05, B= .9 for the cases py = -1, pı = -15,.20,.25; 
Do = .05, py = -10,-15,.20,.25; Po = O1, pı = 02,.05,.10, 15, 20. 
Gi) Compare this with the sample size required if the inspection is by 
attributes and the test is based on the total number of defectives. 


Two-sided t-test. 


(i) Let X,,...,X, be a sample from N(¢, o?). For testing £ = 0 against 
+0, there exists a UMP invariant test with respect to the group 
X! = cX,, c + 0, given by the two-sided t-test (17) of Chapter 5. 

Gi) Let X,,..., X, and Y,,..., Y, be samples from N(£, o°) and N(m, o°) 
respectively. For testing 7 = £ against  * £ there exists a UMP in- 
variant test with respect to the group X! =aX +b, Y; =aY + b, 
a + 0, given by the two-sided t-test (30) of Chapter 5. 

[G): Sufficiency and invariance reduce the problem to |t, which in the notation 

of Section 4 has the probability density s(t) + ps’) for t >0. The Tatio of 

this density for à — ô, to its value for 8 — 0 is proportional to fj^ (e^ + 

e^ 4) g; (v) do, which is an increasing function of 1? and hence of ||.] 
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Testing a correlation coefficient. Let (X,, Y)... (X, Yn) be a sample from a 
bivariate normal distribution. 


(i) For testing p < po against p > pọ there exists a UMP invariant test with 
respect to the group of all transformations X/ = aX, + b, Y’ = cY; + d 
for which a, c > 0. This test rejects when the sample correlation coeffi- 
cient R is too large. 

(ii) The problem of testing p = 0 against p # 0 remains invariant in addition 
under the transformation Y/ = — Y,, X/ = X;. With respect to the group 
generated by this transformation and those of (i) there exists a UMP 
invariant test, with rejection region |R| > C. 


[@: To show that the probability density p,(r) of R has monotone likelihood 
ratio, apply the condition of Chapter 3, Problem 8(i), to the expression (88) 
given for this density in Chapter 5. Putting 1 = pr + 1, the second derivative 
8? log p,(r)/dp ðr up to a positive factor is 


Y «gi [u - (0-9 ++] 


i,j=0 


a z 
21» «| 


i-0 


To see that the numerator is positive for all ¢ > 0, note that it is greater than 


2Y en Y ci - PE- + G7). 


i-0 j=i+1 


Holding i fixed and using the inequality c,,, < $c, the coefficient of t/ in the 
interior sum is > 0.] 


For testing the hypothesis that the correlation coefficient p of a bivariate 
normal distribution is < pọ, determine the power against the alternative 
p = p, when the level of significance a is .05, pọ = .3, p, = .5, and the sample 
size n is 50,100,200. 


Section 5 
Almost invariance of a test $ with respect to the group G of either Problem 
6(i) or Example 6 implies that $ is equivalent to an invariant test. 
Section 6 
Show that 
(i) G, of Example 11 is a group; 
(i) the test which rejects when X2,/X2, > C is UMP invariant under G,; 
(ii) the smallest group containing G, and G, is the group G of Example 11. 
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14. 


15. 


16. 


its 


18. 


Consider a testing problem which is invariant under a group G of transforma- 
tions of the sample space, and let € be a class of tests which is closed under G, 
so that $ € € implies ¢g € €, where $g is the test defined by g(x) = 
(gx). If there exists an a.e. unique UMP member ¢ of €, then $, is almost 
invariant. 


Envelope power function. Let S(a) be the class of all level-a tests of a 
hypothesis H, and let B*(@) be the envelope power function, defined by 


Bx(0) = sup &,(0), 


$€S(a) 


where f, denotes the power function of $. If the problem of testing H 
is invariant under a group G, then B*(0) is invariant under the induced 
group G. 


(i) A generalization of equation (1) is 


[G9 aula) = f f(s) Pyl): 


(ii) If Py, is absolutely continuous with respect to Pj, then Pz», is absolutely 
continuous with respect to Pz», and 


Ps, dPss, 

— = o (gp ae. Py). 

ap, > dPjs, Cn) ak 5.) 

Gii) The distribution of dP} /dP, (X) when X is distributed as Py, is the 
same as that of dP;, / dPjg, (X^) when X’ is distributed as P;,. 

Invariance of likelihood ratio. Let the family of distributions = { Py, 0 € Q} 


be dominated by ji, let pp = dPj/dp, let wg’ be the measure defined by 
ug '(A) = u[g (A), and suppose that p is absolutely continuous with 


respect to pg” ! for all g € G. 
(i) Then 


d 
Po(x) = pu) src (8) (a.e. p). 


Gi) Let Q and w be invariant under G, and countable. Then the likelihood 
ratio supo po (x)/sup, po (x) is almost invariant under G. 


"m = 4 ý ; Jd 
(iii Suppose that p(x) is continuous in 0 for all x, that Q is a separal 
pseudometric space, and that €) and w are invariant. Then the likelihood 


ratio is almost invariant under G. 


Inadmissible likelihood-ratio test. In many applications in which a UMP 
invariant test exists, it coincides with the likelihood-ratio test. That this is, 
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however, not always the case is seen from the following example. Let P,,..., P, 
be n equidistant points on the circle x? + y? — 4, and Q,,...,Q, on the 
circle x? + y? = 1. Denote the origin in the (x, y) plane by O, let 0 < a < i 
be fixed, and let (X, Y) be distributed over the 2n + 1 points 
P,,..., Py, Qi... Qn» O with probabilities given by the following table: 


| ar, o; o 
H a/n (1 — 2a)/n a 
K p,/n 0 (n — 1)/n 


where X. p, = 1. The problem remains invariant under rotations of the plane by 
the angles 2kz/n (k-0,1,...,n — 1). The rejection region of the 
likelihood-ratio test consists of the points P,,..., P,, and its power is 1/n. On 
the other hand, the UMP invariant test rejects when X — Y — 0, and has 
power (n — 1)/n. 


Let G be a group of transformations of 2’, and let be a o-field of subsets of 
4, and u a measure over (£, £). Then a set A €» is said to be almost 
invariant if its indicator function is almost invariant. 


(i) The totality of almost invariant sets forms a o-field .;, and a critical 
function is almost invariant if and only if it is »;-measurable. 

(ii) Let P = {P}, 0 € Q} be a dominated family of probability distributions 
over (X, #), and suppose that g0 = 0 for all g € G, 0 € Q. Then the 
o-field X of almost invariant sets is sufficient for 2. 


[Let A = Lc, P, be equivalent to 2. Then 


dP, aP,-15 dP, 
due E plac odere sided 


so that dP,/4 is almost invariant and hence ,-measurable.] 


Section 7 
The definition of d-admissibility of a test coincides with the admissibility 


definition given in Chapter l, Section 8 when applied to a two-decision 
procedure with loss 0 or 1 as the decision taken is correct or false. 


(i The following example shows that a-admissibility does not always imply 
d-admissibility. Let X be distributed as U(0, 8), and consider the tests qi 
and p, which reject when respectively X < 1 and X « 3 for testing 
H:0-—2 against K:0—1. Then for a=}, p) and p, are both 
a-admissible but q; is not d-admissible. 

(i) Verify the existence of the test p) of Example 12. 
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22. 
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(i) The acceptance region T,/ VB < C of Example 13 is a convex set in the 
(T,, T) plane. 
(ii) In Example 13, the conditions of Theorem 8 are not satisfied for the sets 
A:T,/ /T, x C and Q':£» k. 
(i) In Example 13 (continued) show that there exist C), C, such that Ao(q) 
and A, (n) are probability densities (with respect to Lebesgue measure). 
Gi) Verify the densities hy and hy. 
Verify 


(i) the admissibility of the rejection region (22); 

(ii) the expression for I(Z) given in the proof of Lemma 3. 

Let X,,..., X,; Y,,...,Y, be independent N(£, o?) and N(n, o?) respec- 
tively. The one-sided t-test of H:8 = £/a <0 is admissible against the 
alternatives (i) 0 < 8 < 8, for any ô, > 0; (ii) 8 > ô, for any ô, > 0. 


For the model of the preceding problem, generalize Example 13 (continued) to 
show that the two-sided r-test is a Bayes solution for an appropriate prior 
distribution. 


Section 9 
Wilcoxon two-sample test. Let U,, = 1 or 0 as X, « Y, or X,» Y, and let 
U = EEU, be the number of pairs Xj, Y, with X, < Y 
(i) Then U 2 ES, — }n(n + 1), where Sj < +>: < S, are the ranks of the 
Y's, so that the test with rejection region U > C is equivalent to the 


Wilcoxon test. 

Gi) Any given arrangement of x's and y's can be transformed into the 
arrangement x... xy... y through a number of interchanges of neighbor- 
ing elements. The smallest number of steps in which this can be done for 
the observed arrangement is mn — U. 


Expectation and variance of Wilcoxon statistic. If the X’s and Y's are samples 
from continuous distributions F and G respectively, the expectation and 
variance of the Wilcoxon statistic U defined in the preceding problem are 


given by 

U 
5 —)=P{x< Y} = | FdG 
en Ese eu 
and 


(58) mn Var] = frac + (n - »fa - G) dF 


&(m- 1) fFd6 - (m*n- (frac). 
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Under the hypothesis G = F, these reduce to 


k3 5 U 1 y U mtn+1 
3 (2-3: af) l2mn ` 


(i) Let Z,,...,Zy be independently distributed with densities f,,..., fys 
and let the rank of Z, be denoted by T;. If f is any probability density 
which is positive whenever at least one of the f, is positive, then 


1 Y AY) bs. fn (Va) 
NV Dg) fu) Y 


(6) | P(n 7 t... Ty = tv} = 


where V, < *:- < Ky, is an ordered sample from a distribution with 
density f. 
(estt) N Siid Ba famen snm fifa m tomsa 8 and 


Sı < -+ < S, denote the ordered ranks of Z,,,,,..., Zm4n among all 
the Z's, the probability distribution of S,,..., S, is given by (25). 


{@: The probability in question is f... ffi(z;)... fy(Zy) dz -.. dzy in- 
tegrated over the set in which z, is the ¢,th smallest of the z’s for i = 1,..., N. 
Under the transformation w, =z; the integral becomes f... ffi(w,) 
fy (Wy) dw, ... dwy, integrated over the set w, < --- < wy. The desired 
result now follows from the fact that the probability density of the order 
statistics Wy) <--> < Way is Nlf(w) >>> f(wy) for w< +--+ < wy 


(i) For any continuous cumulative distribution function F, define F~'(0) = 
=œ, F-!(y) — inf(x: F(x) 2 y) for0<y <1, F^!(1) = oo if F(x) 
< 1 for all finite x, and otherwise inf{ x: F(x) = 1). Then F[F ! (y) 
= y forall 0 < y < 1, but F^'[F(y)] may be < y. 

(ii) Let Z have a cumulative distribution function G(z) = A[F(z)], where F 
and h are continuous cumulative distribution functions, the latter de- 
fined over (0,1). If Y = F(Z), then P(Y < y) =h(y) for all 0 < y <1. 

(iii) If Z has the continuous cumulative distribution function F, then F(Z) 
is uniformly distributed over (0, 1). 


(à): PLFCZ) < y)  P(Z < F1) = FIF) = y 


Let Z; have a continuous cumulative distribution function F, (i = L,..., N), 
and let G be the group of all transformations Z/ = f(Z,) such that f is 
continuous and strictly increasing. 


(i) The transformation induced by f in the space of distributions is F; = 
Ff. 

(ii) Two N-tuples of distributions (F,,..., Fy) and (F/,..., Fy) belong to 
the same orbit with respect to G if and only if there exist continuous 
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distribution functions /,..., hy defined on (0,1) and strictly increasing 
continuous distribution functions F and F’ such that F, = h,(F) and 
F! = h,(F). 


(à: PLZ) sy} = P(Z < f O) = RI O). 

(ii): If F, = h,(F) and the F/ are on the same orbit, so that F/ = F( f>), 
then F’ = h,(F’) with F’ = F(f~'). Conversely, if F, = h,(F), F/ =h,(F'), 
then F/ = F( f ) with f= F (F)] 

Under the assumptions of the preceding problem, if F, = h,(F), the distribu- 
tion of the ranks 7;,..., Ty of Z;,...,Zy depends only on the h;, not on F. 
If the h, are differentiable, the distribution of the T, is given by 


E| (Um) (0) 
(61) PITE or Se oni Ta mE 
where Uy) < +++ < Uy) is an ordered sample of size N from the uniform 
distribution U(0, 1). 
[The left-hand side of (61) is the probability that of the quantities 
F(Z,),..., F(Zy), the ith one is the tth smallest for i = 1,..., N. This is 
given by f... fhi (yi)... Ry (yw) dy integrated over the region in which y, is 
the ¢,th smallest of the y's for i — 1,..., N. The proof is completed as in 
Problem 29.] 


Distribution of order statistics. 


(i) If Z,,...,Zy is a sample from a cumulative distribution function F 
with density f, the joint density of Y, = Zis)» i = 15.257,05 


(60 x DKs 8 - DIN - s)! 


xf FO) FO) - F0] B - F(3)] 


for j € = €» 
(i) For the particular case that the Z's are a sample from the uniform 
distribution on (0, 1), this reduces to 


N! 
(63) (0G 5 - Dr. Qr 7$)! 


N-s, 


yr: SpE S07») 


B, w-siis which 


= i density of the beta-distribution 
oriee 4 from U(0, 1). 


therefore is the distribution of the single order statistic Z,,) 
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Gii) Let the distribution of Y,,..., Y, be given by (63), and let V be defined 
by Y, = VV,,,... V, for i ^ 1,..., n. Then the joint distribution of the 
V, is 


N! U S g, 5-1 
Pad alin gon) 


(5,33 =N+1), 
so that the V are independently distributed according to the beta-distri- 
bution B, 


Si Sia TSi" 


10: If Y, 2 Z,,,..., Y, 7 Zip and Y,,,,..., Yy are the remaining Z's in 
the original order of their subscripts, the joint density or yas Y us NCOY T 
1)...(N n * Df... ff nsi f (yw) dy 41. dyy integrated over the re- 
gion in which s, — 1 of the y’s are < y,, s; — s, — 1 between y, and y,..., 
and N — s, > y,. Consider any set where a particular s, — 1 of the y's is 
< yy, a particular s, — s, — 1 of them is between y, and y;, and so on. There 
are N!/(s, — 1)!...(N — s,)! of these regions, and the integral has the same 
value over each of them, namely [F(yi)]  ! [F(y;) — F(y]?^ ^^... — 


F(y" 7] 


34. (i) If X,..., X, and Y,...,Y, are samples with continuous cumulative 
distribution functions F and G = h(F) respectively, and if / is differen- 
tiable, the distribution of the ranks S} < --- < S, of the Y's is given by 


("ms") 


where Uy) < ::: < U,,,,, is an ordered sample from the uniform 
distribution U(0, 1). 


(i) If in particular G = F^, where k is a positive integer, (64) reduces to 


(64) P{S, =5,,...,5,=5,} = 


(65) P{S, = 5,,...,5,=5,} 
TON ae tl T(s*jk-j) TP (sj1) 
iren) jut T(s;) T(s; tjk-j) 


35. For sufficiently small 6 > 0, the Wilcoxon test at level 
a= «Jf N) " k a positive integer, 


maximizes the power (among rank tests) against the alternatives (F, G) with 
= (1 —0)F + 6F.. 
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36. 


E 


38. 


39. 


An alternative proof of the optimum property of the Wilcoxon test for 
detecting a shift in the logistic distribution is obtained from the preceding 
problem by equating F(x — 0) with (1 — 0) F(x) + F^(x), neglecting powers 
of 0 higher than the first. This leads to the differential equation F — 0F' = 
(1 — 0) F + 6F?, the solution of which is the logistic distribution. 


Let 4, be a family of probability measures over (2, 3), and let € be a class 
of transformations of the space Z. Define a class ¥, of distributions by 
F, € F, if there exists F) € A and f € € such that the distribution of f(X) 
is F, when that of X is F. If $ is any test satisfying (a) Ep $ (X) = a for all 
F, € 4, and (b) $(x) < $[f(x)] for all x and all f E €, then $ is unbiased 
for testing A, against F}. 

Let X,,..., Xm} Y,,..., Y, be samples from a common continuous distribution 
F. Then the Wilcoxon statistic U defined in Problem 27 is distributed symmet- 
rically about 1mn even when m # n. 


(i) If X,,..., X, and Y,,..., Y, are samples from F(x) and G(y) = F(y 
— A) respectively (F continuous), and Dy, < +++ < Demn) denote the 
ordered differences Y, — X,, then 


P[Duy «^« Dosi-o] = Polk s Us m - k], 


where U is the statistic defined in Problem 27 and the probability on the 
right side is calculated for A = 0. 
(i) Determine the above confidence interval for A when m=n= 6, 
the confidence coefficient is 22, and the observations are x: 
.113, .212, .249, 522, .709, .788, and y: 221, 433, .724, .913, .917, 1.58. 
(iii) For the data of (ii) determine the confidence intervals based on Student’s 
t for the case that F is normal. 


[@: D, < A < Dixy if and only if Uy = mn — i, where U, is the statistic U 
of Problem 27 calculated for the observations 


X395 X, y ey cw] 


G) Let X, X’ and Y, Y' be independent samples of size 2 from continuous 
distributions F and G respectively. Then 


p = P(max(X, X^) < min(Y, Y) + P(max(Y, Y) < min( X, X’)} 
=} +24, 


where A = f(F — G} d[(F + G)/2. 
(ii) A = 0 if and only if F = G. 
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IG): p = JQ — F)? dG? + JG — G)? dF’, which after some computation re- 
duces to the stated form. 5 

(ii); A = 0 implies F(x) = G(x) except on a set N which has measure zero 
both under F and G. Suppose that G(x,) — F(x,) =n > 0. Then there exists 
xo such that G(x) = F(x) + łn and F(x) < G(x) for xy < x < x,. Since 
G(x,) — G(xo) > 0, it follows that A > 0.] 


Continuation. 


(i) There exists at every significance level a atest of H: G = F which has 
power > a against all continuous alternatives (F, G) with F + G. 


(ii) There does not exist a nonrandomized unbiased rank test of H against 
all G + F at level 
—1/(mtn 
a= T ( on ). 


{(i); let X, X/; Y, ¥/ (i = 1,..., n) be independently distributed, the X's with 
distribution F, the Y’s with distribution G, and let V, = 1 if max( X,, X/) < 
min(Y,, Y/) or max(Y,, Y/) < min(X,, X/), and V, = 0 otherwise. Then LV; has 
a binomial distribution with the probability p defined in Problem 40, and the 
problem reduces to that of testing p = 3 against p > J. 


(ii): Consider the particular alternatives for which P( X < Y } is either 1 or 0.] 


Section 10 
(i) Let m and n be the numbers of negative and positive observations among 
Z,,..., Zy, and let S} < -++ < S, denote the ranks of the positive Z's 
among |Z;|,..., |Zy| Consider the N + 3N(N — 1) distinct sums Z; + Z; 
with ¿=j as well as i + j. The Wilcoxon signed rank statistic LS; is 
equal to the number of these sums that are positive. 
(ii) If the common distribution of the Z's is D, then 


E(Y.S) =4N(N + 1) - ND(0) — 4N(N — 1) [D(—z) dD(2). 


© Let K be the required number of postive sums. Since Z, + Z; is positive 
if and only if the Z corresponding to the larger of |Z,] and |Z;| is positive, 
K=YN,L}_,U,,, where U; = 1 if Z, > 0 and |Z| < Z; and U, = 0 other- 
wise.] 


Let Z,,..., Zy be a sample from a distribution with density f(z — 0), where 
f(z) is positive for all z and f is symmetric about 0, and let m, n, and the S; 
be defined as in the preceding problem. 


(i) The distribution of n and the S, is given by 

(66) \P{the number of positive Z'sis n and S, = 5,,..., Sa = Sn } 
1 [I9 0s) 0) os ~ 9) 
25 f(Vay)--- FY) 
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where Va, < +++ < yy is an ordered sample from a distribution with 
density 2 f(v) for v > 0, and 0 otherwise. 

(i) The rank test of the hypothesis of symmetry with respect to the origin, 
which maximizes the derivative of the power function at @ = 0 and hence 
maximizes the power for sufficiently small 0 > 0, rejects, under suitable 
regularity conditions, when 


“Ag f(v«)) 


uns fi Ys) 


(ii) In the particular case that f(z) is a normal density with zero mean, the 
rejection region of (ii) reduces to EE(V,)) > C, where Vy € °*° 
< Vy) is an ordered sample from a x-distribution with 1 degree of 
freedom. 

(iv) Determine a density f such that the one-sample Wilcoxon test is 
most powerful against the alternatives f(z — 8) for sufficiently small 
positive 8. 


[(i): Apply Problem 29(i) to find an expression for P(S$; = 5$, Sy = 5, 
given that the number of positive Z's is n)] 

An alternative expression for (66) is obtained if the distribution. of Z is 
characterized by (p, F, G). If then G= h(F) and h is differentiable, the 
distribution of n and the S, is given by 


(67) P(Q- )"E[A’(Usy) ES w(u.,;)]. 


where Uy, € -+ < Uy is an ordered sample from U(0, 1). 


Unbiased tests of symmetry. Let Z,,-.-» Zy be a sample, and let be any 
rank test of the hypothesis of symmetry with respect to the origin such. that 
z, €z| for all i implies (4) zy) $ $(2{, -> ZN) Then $ is unbiased 
against the one-sided alternatives that the Z's are stochastically larger than 
some random variable that has a symmetric distribution with respect to the 
origin. : 

The hypothesis of randomness. Let Z,,.--» Zw be independently distributed 
with pr ne F,,..., Fy, and let T, denote the rank of Z, among the Z's. 
For testing the hypothesis of randomness F, = *'* = Fy against the alterna- 
tives K of an upward trend, namely that Z, is stochastically increasing with i, 
consider the rejection regions 


(68) Lit, > C 


and 
(69) Lie (Kay) > € 
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where Wa < -:: < Vy) is an ordered sample from a standard normal 
distribution and where 1, is the value taken on by T7;. 


(i) The second of these tests is most powerful among rank tests against the 
normal alternatives F = N(y + i8, o?) for sufficiently small ô. 
(ii) Determine alternatives against which the first test is a most powerful 
rank test. 
(iii) Both tests are unbiased against the alternatives of an upward trend; so is 
any rank test $ satisfying $(z,,...,Zy) <$(z{,...,zy) for any two 
points for which i < j, z; < z; implies z/ < zj for all i and j. 


(Gii): Apply Problem 37 with € the class of transformations z{ = z,, z; = f;(z;) 
for i > 1, where z < f,(z) < --- < fy(z) and each f, is nondecreasing. If 9 
is the class of N-tuples (F,,..., Fy) with F = --- = Fy, then F, coincides 
with the class K of alternatives] 


In the preceding problem let U; = 1 if (j — i(Z, — Z,) > 0, and = 0 other- 
wise. 


(i) The test statistic Li7; can be expressed in terms of the U's through the 
relation 


aS Oe he, EINN ED) 
ERU i<j " 6 s 


(ii) The smallest number of steps [in the sense of Problem 27(ii)] by which 
(Z,,..., Zy) can be transformed into the ordered sample (Zi, - -> Zuv)) 
is [N(N — 1)/2] - U, where U —X,.,U;. This suggests U> C as 
another rejection region for the preceding problem. 

(@: Let Vi, = 1 or 0 as Z, < Z; or Z; > Z;. Then T, = EX; and Vj = Ui, 

or 1 — Uj as i <j or i > j. Expressing E^, jT, = E^, JEX iV; in terms of 

the U's and using the fact that U;; = U;,, the result follows by a simple 
calculation.] 


The hypothesis of independence. Let (X, , ¥;),...,(Xy, Yy) be a sample from a 
bivariate distribution, and (Xa), Z;),...,(Xw),Zy) be the same sample 
arranged according to increasing values of the X’s, so that the Z's are à 
permutation of the Y's. Let R, be the rank of X, among the X's, S; the rank 
of Y, among the Y's, and T, the rank of Z, among the Z's, and consider the 
hypothesis of independence of X and Y against the alternatives of positive 
regression dependence. 


© Conditionally, given (X, ..., X, y,), this problem is equivalent to test- 
ing the hypothesis of randomness of the Z’s against the alternatives of 
an upward trend. 
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(ii). The test (68) is equivalent to rejecting when the rank correlation coeffi- 
cient 


X(R-R(s-S) . n 


/E(.-3yE( -8y MN 


is too large. 
(ii) An alternative expression for the rank correlation coefficient* is 


rfr- =s- L| 


6 6 T 
1-armgli a 


(iv) The test U > C of Problem 47(ii) is equivalent to rejecting when Kendall's 
t-statistic* X, < Vi,/N(N — 1) is too large where V,; is +1 or —1 as 
(Y, — Y(X, — X;) is positive or negative. 

(v) The tests (ii) and (iv) are unbiased against the alternatives of positive 
regression dependence. 


Section 11 


49. In Example 16, a family of sets S(x, y) isa class of equivariant confidence sets 
if and only if there exists a set 4? of real numbers such that 


S(x,y) = U (Gm: 7^ + =a =} 


re 


50. Let X,,..., Xp; Yis...» Yp be samples from N(£,o?) and N(n, 7^) respec- 


tively. Then the confidence intervals (43) of Chapter 5 for 7?/o?, which can be 
written as 


Lcid] DUE) 
kX(x-xX) v X(G-X) 


are uniformly most accurate equivariant with respect to the smallest group G 
containing the transformations X/ = aX + b, Y/ = aY + c forall a + 0, b,c 
and the transformation X/ = dY,, Y/ = X,/d for all d#0. 


[Cf. Problem 7.] 
51. (i) One-sided equivariant confidence limits. Let 6 be real-valued, and sup- 


pose that for each 6, the problem of testing 0 = bo against 0 > 6, (in 

the presence of nuisance parameters ®) remains invariant under a group 
*For further material on these statistics see Kendall (1970); Aiyar, Guillier, and Albers 
(1979); and books on nonparametric inference. 
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Gp, and that A(@) is a UMP invariant acceptance region for this 
hypothesis at level a. Let the associated confidence sets S(x) = {0 : x € 
A(0)) be one-sided intervals S(x) = (0:6(x) < 0), and suppose they 
are equivariant under all G, and hence under the group G generated by 
these. Then the lower confidence limits 6( X) are uniformly most accurate 
equivariant at confidence level 1 — «a in the sense of minimizing 
Py 5(0(X) x 0") for all 0’ < 6. 

Let X,...,X, be independently distributed as N(£,o?). The upper 
confidence limits o? < E(X, — X)?/C) of Example 5, Chapter 5, are 
uniformly most accurate equivariant under the group X/ = X; + c, —oo 
< c < œ. They are also equivariant (and hence uniformly most accurate 
equivariant) under the larger group X/ = aX, + c, —oo < a,c < oc. 


Counterexample. The following example shows that the equivariance of S(x) 
assumed in the paragraph following Lemma 5 does not follow from the other 
assumptions of this lemma. In Example 8, let n = 1, let G® be the group G of 
Example 8, and let G® be the corresponding group when the roles of Z and 


Y= 


Y, are reversed. For testing H(05):0 = 0, against 0 + @ let G, be 


equal to G augmented by the transformation Y' = 6) — (Y, — 4) when 
6 < 0, and let Gy, be equal to G® augmented by the transformation Z’ = b 
— (Z — 6j) when 0 > 0. Then there exists a UMP invariant test of H(%) 
under Gp, for each 6,, but the associated confidence sets S(x) are not 
equivariant under G = {G,, —oo < 0 < oo). 


G) 


(ii) 


@ 


Let X,,..., X, be independently distributed as N(£, 07), and let 0 = £/o. 
The lower confidence bounds @ for 0, which at confidence level 1 — a are 
uniformly most accurate invariant under the transformations X/ = aX; 
are 


vnX 
VX GC - XP An- 1) 


wheie the function C(@) is determined from a table of noncentral t 5o 
that 


UC 


YnX 


«C(0)) 21-—a. 
VEX -EAn -1) st 


Determine @ when the x's are 7.6, 21.2, 15.1, 32.0, 19.7, 25.3, 29.1, 184 
and the confidence level is 1 — a — .95. 


Lo 


Let (Xi, Y,),...,(X,, Yp) be a sample from a bivariate normal distribu- 
tion, and let 


p» ca| LEO - 30 - Y) 


" (xa -xyrg-rvy] 
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where C(p) is determined such that 
X(x - X) - Y) 
P, c 
Earm y” 


Then p is a lower confidence limit for the population correlation coeffi- 
cient p at confidence level 1 — a; it is uniformly most accurate invariant 
with respect to the group of transformations X/ = aX; + b, Y/ = cY, + d, 
with ac > 0, —oo « b, d < oo. 

(ii) Determine p at level 1 — a = .95 when the observations are (12.9, .56), 
(9.8, .92), (13.1, 42), (12.5, 1.01), (8.7, .63), (10.7, .58), (9.3, .72), (11.4, .64). 


Section 12 
In Examples 20 and 21 there do not exist equivariant sets that uniformly 
minimize the probability of covering false values. 
In Example 20, the density p(v) of V = 1/S? is unimodal. 
Show that in Example 20, 


(i) the confidence sets o?/S? € A** with A** given by (40) coincide with 
the uniformly most accurate unbiased confidence sets for o?; 

(ii) if (a, b) is best with respect to (39) for c, then (a, b^) is best for o^ 
(r > 0). 

Let X,,..., X, be independent N(0,1), and let S? be pendent of the X's 

and distributed as x2. Then the distribution of (X,/SV» ,..., X,/Syv) is a 

central multivariate ¢-distribution, and its density is 


T(i(» + 7)) 1 i -hotn 
Pap e ee ot vi 2 i 


The confidence sets (47) are uniformly most accurate equivariant under the 
group G defined at the end of Example 22. 


In Example 23, show that 


(Gi) both sets (55) are intervals; 

Gi) the sets given by vp (v) > C coincide with the intervals (42) of Chapter 5. 
Let a XG Yop ees independently normally distributed as N(£, o?) 
and N(n, o?) respectively. Determine the equivariant confidence sets for n — £ 
that have smallest Lebesgue measure when 


G) c is known; 
(ii) o is unknown. 
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Generalize the confidence sets of Example 18 to the case that the X, are 
N(£,, d;o?) where the d’s are known constants. 


Solve the problem corresponding to Example 20 when 

(i) X,..., X, is a sample from the exponential density E($, o), and the 
parameter being estimated is o; 

(i) X,,...,X, is a sample from the uniform density U(£, £ * 7), and the 
parameter being estimated is 7. 


Let X,,..., X, be a sample from the exponential distribution E(£, c). With 
respect to the transformations X/ = bX, + a determine the smallest equiv- 
ariant confidence sets 


(i) for ø, both when size is defined by Lebesgue measure and by the 
equivariant measure (39); 

(i) for &. 
Let X,, (J = 1,..-)m); i = 1,...,) be samples from the exponential distribu- 
tion E(£,, o). Determine the smallest equivariant confidence sets for (£),..-, $,) 
with respect to the group X/; = bX;; + aj. 

Section 13 
If the confidence sets S(x) are equivariant under the group G, then the 


probability P,(0 € S(X)) of their covering the true value is invariant under 
the induced group G. 


Consider the problem of obtaining a (two-sided) confidence band for an 
unknown continuous cumulative distribution function F. 


(i) Show that this problem is invariant both under strictly increasing and 
strictly decreasing continuous transformations X/ = f(X;), i = Ll..." 
and determine a maximal invariant with respect to this group. 

(i) Show that the problem is not invariant under the transformation 


X, if |X|2 1, 
X/-(X-1 if 0«X«1, 
X,+1 if -1«X, «0. 
[(ii): For this transformation g, the set g*S(x) is no longer a band.] 


Additional Problems 
Let X,,..., X, be a sample from a distribution with density 


TORE) 
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where f(x) is either zero for x <0 or symmetric about zero. The most 
powerful scale-invariant test for testing H: f — fọ against K:f — f, rejects 
when 


T CORETCOE 
>C 
[Por hlo) oon) dv 


69. Normal vs. double exponential. For fj (x) = e 7s, f(x) =e 72, 

the test of the preceding problem reduces to rejecting when yxx? /Elx,| < C. 

(Hogg, 1972.) 

Note. The corresponding test when both location and scale are unknown is 

obtained in Uthoff (1973). Testing normality against Cauchy alternatives is 
discussed by Franck (1981). 


70. Uniform vs. triangular. 


(i) Forf(x)-1(0«x«1), f(x) -2x(0«x < 1), the test of Problem 
68 reduces to rejecting when T = x(,,/x < C. 
(i) Under fo, the statistic 2n log T is distributed as xi, 


(Quesenberry and Starbuck, 1976.) 
71. Show that the test of Problem 5(i) reduces to 


G) [xg — xaj1/S < c for normal vs. uniform; 
Gi) [X — xq) ]/S < c for normal vs. exponential; 
(i) [Xx — xa JAX) — Xa < € for uniform vs. exponential. 


(Uthoff, 1970.) 


Note, When testing for normality, one is typically not interested in distinguishing 
the normal from some other given shape but would like to know more generally 
whether the data are or are not consonant with a normal distribution. This is a 
special case of the problem of testing for goodness of fit, briefly referred to at the 
end of Section 13. Methods particularly suitable for testing normality are discussed 
for example in Shapiro, Wilk, and Chen (1968), Hegazy and Green (1975), 
D'Agostino (1982), Hall and Welsh (1983), and Spiegelhalter (1983), and for testing 
exponentiality in Galambos (1982), Brain and Shapiro (1983), Spiegelhalter (1983), 
Deshpande (1983), Doksum and Yandell (1984), and Spurrier (1984). See also Kent 
and Quesenberry (1982). 

72. The UMP invariant test of Problem 69 is also UMP similar. 


[Consider the problem of testing a — 0 vs. «> 0 in the 
exponential family with density 


two-parameter 


a l-a 
Cla, ren -73 E- = Ek). 0<a<1] 
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Note. For the analogous result for the tests of Problem 70, 71, see 
Quesenberry and Starbuck (1976). 


The following UMP unbiased tests of Chapter 5 are also UMP invariant under 
change in scale: 


(i The test of g < go in a gamma distribution (Problem 73 of Chapter 5). 
(i) The test of b, < b; in Problem 75(i) of Chapter 5. 


Let X,,..., X, be a sample from N(£, 0?), and consider the UMP invariant 
level-a test of H : £/a < 6, (Section 6.4). Let a, (F) be the actual significance 
level of this test when X;,..., X, is a sample from a distribution F with 
E(X,) =£, Var(X)) = o? < oo. Then the relation a,(F) —^ a will not in 
general hold unless 6, = 0. 

[Use the fact that the joint distribution of yn (X —£) and yn (S^ — o?) tends 
to the bivariate normal distribution with mean zero and covariance matrix 


o? DEI 

Hy |4— o2], 
where S? = E(X, — X)?/n and p, = E(X; — £)*. See for example Serfling 
(1980)] 


The totality of permutations of K distinct numbers 4,,...,ay for varying 
4,,...,ay Can be represented as a subset Cy of Euclidean K-space Rx, and 
the group G of Example 8 as the union of C,,C,,.... Let » be the measure 
over G which assigns to a subset B of G the value Yu (B O Cx), where 
fx denotes Lebesgue measure in Ey. Give an example of a set B C G and an 
element g € G such that »(B) > 0 but »(Bg) = 0. 

[If a, b, c, d are distinct numbers, the permutations g, g’ taking (a, b) into 
Gun and (c, d) into (d, c) respectively are points in C,, but gg’ is a point in 

d 


The Kolmogorov test (56) for testing H : F = F, (F, continuous) is consistent 
against any alternative F; * F}, that is, its power against any fixed F, tends to 
las n> oo. 

[The critical value A = A, of (56) corresponding to a given a satisfies ynh >K 
for some K > 0 as n — oo. Let a be any value for which F,(a) + Fy(a), and 
use the facts that (a) |Fy(a) — Ty(a)| < sup|Fy(u) — Ty (u)| and (b) if F = Fi» 
the statistic Ty (a) has a binomial distribution with success probability p = 
F(a) * Fy(a).] [Massey (1950).] 
Note. For exact power calculations in both the continuous and discrete case, 
see for example Niederhausen (1981) and Gleser (1985). 


G) Let Xis...» Xm; Y,,..., Y, be iid, according to a continuous distribution 
F, let the ranks of the Y's be S, < --- <S,, and let T= A(5) 
+--+ +h(S,). Then if either m =n or h(s) + h(N + 1 — s) is inde- 
pendent of s, the distribution of T is symmetric about nEX A (i)/N- 


i-1 
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(ii) Show that the two-sample Wilcoxon and normal-scores statistics are 
symmetrically distributed under H, and determine their centers of sym- 
metry. 


[G): Let S/ = N +1 -— S,, and use the fact that 7" = Eh(S/) has the same 
distribution under H as T.] 

Note. The following problems explore the relationship between pivotal quan- 
tities and equivariant confidence sets. For more details see Arnold (1984). 

Let X be distributed according P, ;, and consider confidence sets for 0 that 
are equivariant under a group G*, as in Section 11. If w is the set of possible 
0-values, define a group G on 4X w by (0, x) = (gx, 20). 


78. Let V( X, 0) be any pivotal quantity [i.e. have a fixed probability distribution 
independent of (0, 9)], and let B be any set in the range space of V with 
probability P(V € B) — 1 — a. Then the sets S(x) defined by 


(70) @€S(x) ifandonlyif V(0,x) € B 


are confidence sets for 0 with confidence coefficient 1 — a. 


79. (i) If G is transitive over 4 X w and V(X, 0) is maximal invariant under G, 
then V( X, 0) is pivotal. " 
(i) By (i), any quantity W(X, 8) which is invariant under G is pivotal; give 
an example showing that the converse need not be true. 


80. Under the assumptions of the preceding problem, the confidence set S(x) is 
equivariant under G*. 

81. Under the assumptions of Problem 79, suppose that a family of confidence sets 
S(x) is equivariant under G*. Then there exists a set B in the range space of 
the pivotal V such that (70) holds. In this sense, all equivariant confidence sets 
can be obtained from pivotals. 

[Let A be the subset of ZX w given by A = {(x, 0): 0 € S(x)). Show that 
ŽA = A, so that any orbit of G is either in A or in the complement of A. Let 
the maximal invariant V(x, 0) be represented as in Section 2 by a uniquely 
defined point on each orbit, and let B be the set of these points whose orbits 
are in A. Then V(x, 0) € B if and only if (x, 0) € A.] 

Note. Problem 80 provides a simple check of the equivariance of confidence 
sets. In Example 21, for instance, the confidence sets (41) are based on the 
pivotal vector ( X, — £&j,..., X, — §,), and hence are equivariant. 
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QHA PER 


Linear Hypotheses 


1. A CANONICAL FORM 


Many testing problems concern the means of normal distributions and are 
special cases of the following general univariate linear hypothesis. Let 
X,..., X, be independently normally distributed with means ¢,,..., £, 
and common variance o?. The vector of means* £ is known to lie in a given 
s-dimensional linear subspace Io (s < n), and the hypothesis H is to be 
tested that £ lies in a given (s — r)-dimensional subspace II, of Ig 
(rx s). 3 


Example 1. In the two-sample problem of testing equality of two normal means 
(considered with a different notation in Chapter 5, Section 3), it is given that £, = £ 
for i — 1,..., n, and £, = q for i = nj + 1,..., n, + n5, and the hypothesis to be 
tested is n = £. The space Io is then the space of vectors 


(Eht mena 9 E(0,...,1,0,...,0) * 0(0,...,0,1,...,1) 


spanned by (1,...,1,0,...,0) and (0,...,0,1,...,1), so that s = 2. Similarly, II, is 
the set of all vectors (£,..., £) = £(1,..., 1), and hence r = 1. 

Another hypothesis that can be tested in this situation is 7 = ¿= 0. The space 
IL, is then the origin, s — r = 0 and hence r = 2. The more general hypothesis 
È= £5, n = nj is not a linear hypothesis, since II, does not contain the origin. 


However, it reduces to the previous case through the transformation X/ = X, — £o 
(i= 1,...,n), X/ =X, ^ 99 (io m * 1... m +n). 


. Example 2. The regression problem of Chapter 5, Section 8, is essentially a 
linear hypothesis. Changing the notation to make it conform with that of the present 
Section, let £, = a + Bt,, where a, ß are unknown, and the /, known and not all 
equal. Since Io is the space of all vectors a(1,...,1) + B(t,.--,%)) it has 
dimension s = 2. The hypothesis to be tested may be a — B — 0 (r= 2) orit may 


"Throughout this chapter, a fixed coordinate system is assumed given in n-space. A vector 
with components £,,...,£, is denoted by $, and an n X 1 column matrix with elements 


Hume s by £. 
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only specify that one of the parameters is zero (r = 1). The more general hypotheses 
a = ap, B = Bo can be reduced to the previous case by letting X/ = X, — % — Botis 
since then E(X/) = a’ + B't; with a^ = a — a, B' = B — Bs. 

Higher polynomial regression and regression in several variables also fall under 
the linear-hypothesis scheme. Thus if $; = a + Bt; + yt? or more generally £, = a 
+ Bt, + yu;, where the t, and u; are known, it can be tested whether one or more 
of the regression coefficients a, B, y are zero, and by transforming to the variables 
X! — ay — Bot; — You; also whether these coefficients have specified values other 
than zero, 


In the general case, the hypothesis can be given a simple form by making 
an orthogonal transformation to variables Y,,..., Y, 


n 


(1) Y-CX, C=(c,) i,j=1,....n, 

such that the first s row vectors c;,..., c, of the matrix C span Ig, with 
Cip €, Spanning I. Then Y,,, = --- = Y, = 0 if and only if X is 
in Mg and Y= --- = Y,= Y= --: = ¥,=0 if and only if X 


is in IL,. Let n; = E(Y;), so that n = C£. Then since £ lies in Ig a priori 
and in II, under H, it follows that n,=0 for i=s+1,..., in both 
cases, and y,=0 for i— L,...,r when H is true. Finally, since the 
transformation is orthogonal, the variables Y,,..., Y, are again indepen- 
dently normally distributed with common variance o?, and the problem 
reduces to the following canonical form. 

The variables Y,,..., Y, are independently, normally distributed with 
common variance c? and means E(Y;) = n; for i = 1,...,s and E(Y;) - 0 
for i =s + 1,..., n, so that their joint density is 


$ 1 E 5 n 3 
Q) aepo Eo 9 * Y 2 


i=l i=s+1 


The 1’s and o? are unknown, and the hypothesis to be tested is 
(3) Hop eese EQ T (rS s cn). 


Example 3. To illustrate the determination of the transformation (1), consider 
once more the regression model £, = a + t; of Example 2. It was seen there that 
IIg is spanned by (1,...,1) and (1,,..., t4). If the hypothesis being tested is B = 0, 
IL, is the one-dimensional space spanned by the first of these vectors. The row 
vector c; is in IL, and of length 1, and hence c; = (1/V/n,...,1/ yn). Since c; is 
in Ig, of length 1, and orthogonal to c;, its coordinates are of the form a + bti, 
i=1,...,n, where a and b are determined by the conditions X(a + bt;) = 0 and 


X(a + bt)? = 1. The solutions of these equations are a= —bi, b= 


7.1] . A CANONICAL FORM 367 
1/ VE (4, — 7)”, and therefore a + bt, = (1, — i)/ X(t, — i)’, and 
LEX = 1)  X(X-X).-i) 

yE-0  Yqdmy-y c 


The remaining row vectors of C can be taken to be any set of orthogonal unit 
vectors which are orthogonal to Tg; it turns out not to be necessary to determine 
them explicitly. 

If the hypothesis to be tested is a = 0, II, is spanned by (t,,..., 1,,), so that the 
ith coordinate of c, is t,/ yEg . The coordinates of c, are again of the form 


a+ bt, with a and b now determined by the equations D(a + bt,)t, = 0 and 
Y(a + bt,)? — 1. The solutions are b = —ani/Y6, a = VEt?/nY(t — 7)’, and 


therefore 
DM (x i | 
Y, 2| —————À|X-—Xx|. 
^p EG-YX (ES 


In the case of the hypothesis a = f = 0, II, is the origin and £1, €; can be taken as 
any two orthogonal unit vectors in Ig. One possible choice is that appropriate to 
the hypothesis 8 = 0, in which case Y, is the linear function given there and Y, 


— yn X. 


The general linear-hypothesis problem in terms of the Y's remains 
invariant under the group G, of transformations Y/ = Y, + c, for i = r + 
1,..., 8; Y/ = Y, for i = 1... 7; 5+ 1,..., n. This leaves ¥;,..., Y, and 
Y,,1,--., Y, as maximal invariants. Another group of transformations leav- 
ing the problem invariant is the group G, of all orthogonal transformations 
of Y,,..., Y,. The middle set of variables having been eliminated, it follows 
from Chapter 6, Example l(ii) that a maximal invariant under e, is 
U = Xr Y, Y, ,,..., Yp. This can be reduced to U and V = T aby 
Sufficiency. Finally, the problem also remains invariant under the group G, 
of scale changes Y/ = cY, c + 0, for i = 1,..., n. In the space of U and V 
this induces the transformation U* = c?U, V* = c?V, under which W = 
U/V is maximal invariant. Thus the principle of invariance reduces the data 


to the single statistic* 


Y, 


(4) w= NM 


*A corresponding reduction without assuming normality is discussed by Jagers (1980). 
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Each of the three transformation groups G; (i = 1, 2, 3) which lead to the 
above reduction induces a corresponding group G, in the parameter space. 
The group G, consists of the translations n; = 9; + c; (i — r + L...,5), 


m, =n; (i9 l...,r), o = o, which leaves (7;,..., n, 0) as me in- 
variants, Since any orthogonal transformation gi Y,,..-, Y, induces the 
same transformation on 7),.- D and leaves o? unchanged, a maximal 


invariant under G, is Œi- ot ). Finally the elements of G, are the 
transformations wj = cn, o' = |c|o, and hence a maximal invariant with 
respect to the totality of these transformations is 


Lr 

i=1 
(5) y= WT 
It follows from Theorem 3 of Chapter 6 that the distribution of W depends 
only on y, so that the principle of invariance reduces the problem to that 
of testing the simple hypothesis H : y = 0. More precisely, the probability 
density of W is (cf. Problems 2 and 3) 


oo i2) wir itk 

6 mpi. Lasse a tl 
(6) p(w)=e ue Gawker 
where 


T(r +n-s) +k] 
T (1r k)r[1(n -s)| 


€, — 


For any /, the ratio p, (w)/po(w) is an increasing function of w, and it 
follows from the Neyman-Pearson fundamental lemma that the most 
powerful invariant test for testing y = 0 against y = y, rejects when W is 
too large, or equivalently when 


3 Y?/r 
(7) we = —-" ——_>c. 
2: e/a s) 


isl 


The cutoff point C is determined so that the probability of rejection is A 
when y = 0. Since in this case W* is the ratio of two independent Xa 
variables, each divided by the number of its degrees of freedom, the 
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distribution of W* is the F-distribution with r and n—s degrees of 
freedom, and hence C is determined by 


(8) [ Fac) dro a. 


The test is independent of y, and hence is UMP among all invariant tests. 
By Theorem 5 of Chapter 6, it is also UMP among all tests whose power 
function depends only on 47. 

The rejection region (7) can also be expressed in the form 


(9) —————M—— s 

25 Vitis Xj. 

i=l i=s+1 
When y = 0, the left-hand side is distributed according to the beta-distribu- 
tion with r and n — s degrees of freedom [defined through (24) of Chapter 
5], so that C” is determined by 


(10) By io) D = a. 
Cc 273 1 


For an alternative value of y, the left-hand side of (9) is distributed 
according to the noncentral beta-distribution with noncentrality parameter 
Y, the density of which is (Problem 3) 


eU) 
(11) g,(») =e X Bart Maman )- 
k-o ^ 
The power of the test against an alternative y is therefore* 
1 
BU) = f 80) dv. 
In the particular case r = 1, the rejection region (7) reduces to 
Yi 
(12) - | il e (ely 
X raS) 
i-stl 


"Tables of the power of the F-test are provided by Tiku (1967, 1972) (reprinted in Graybill 
(1976)] and Cohen (1977); charts are given in Pearson and Hartley (1972). Various approxima- 
tions are discussed by Johnson and Kotz (1970). 
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This is a two-sided t-test, which by the theory of Chapter 5 (see for example 
Problem 5 of that chapter) is UMP unbiased. On the other hand, no UMP 
unbiased test exists for r > 1. 

The F-test (7) shares the admissibility properties of the two-sided t-test 
discussed in Chapter 6, Section 7. In particular, the test is admissible against 
distant alternatives 4? > i? (Problem 6) and against nearby alternatives 
y? < 4} (Problem 7). It was shown by Lehmann and Stein (1953) that the 
test is in fact admissible against the alternatives 4? = i for any y, and 
hence against all invariant alternatives. 


2. LINEAR HYPOTHESES AND LEAST SQUARES 


In applications to specific problems it is usually not convenient to carry out 
the reduction to canonical form explicitly. The test statistic W can be 
expressed in terms of the original variables by noting that D’_,,,Y, is the 
minimum value of 


E-a E Y= EIT- EY)? 


i=l i=s+1 i=l 


under unrestricted variation of the ņ’s. Also, since the transformation 
Y = CX is orthogonal and orthogonal transformations leave distances 
unchanged, 


n 


DIU -E(Y)|- X(x-&)y. 


i=l 


Furthermore, there is a 1 : 1 correspondence between the totality of s-tuples 
(m... Ns) and the totality of vectors £ in IIg. Hence 


n n 
(3) Y Y-Y(Qx-&y 
i=st+l i=1 
where the £’s are the least-squares estimates of the £'s under Q, that is, the 


values that minimize Y7..,( X, — £;)? subject to £ in IIo. 
In the same way it is seen that 5 


Ere X y= (x8) 


iar i=s+1 im] 


where the £'s are the values that minimize E(X, — £,)? subject to in IL, 
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Figure 1 


The test (7) therefore becomes 
Y(x-&y-XQc-U' 
Qà) — wee det, = » C 
¥ Or, - &£)/(n - s) 


i=l 


where C is determined by (8). Geometrically the vectors £ and £ are the 
projections of X on IIg and II,, so that the triangle formed by X, §, and 5 
has a right angle at Ê (Figure 1.) Thus the denominator and numerator of 
W*, except for the factors 1/(n — s) and 1/r, are the squares of the 
distances between X and Ê and between £ and £ respectively. An alterna- 
tive expression for W * is therefore 


X (&- £y 


(15) Wt == —. 
Y (x, - &)/(n- 5) 
i=l 


A 27g? 
It is desirable to express also the noncentrality parameter y? = Xim/e 


in terms of the £'s. Now X = C^!Y, £ = C^, and 


(16) X Yp = È(x-ê)- ÈE 


If the right-hand side of (16) is denoted by f(X), it follows that X{-m; = 
K(&). 
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A slight generalization of a linear hypothesis is the inhomogeneous 
hypothesis which specifies for the vector of means £ a subhyperplane IT’, of 
To not passing through the origin. Let II, denote the subspace of Ig 
which passes through the origin and is parallel to II/,. If ° is any point of 
II^, the set II’, consists of the totality of points = £* + £° as £* ranges 
over II,,. Applying the transformation (1) with respect to IT,,, the vector of 
means 7 for £ € II’, is then given by n = C£ = C£* + C£? in the canoni- 
cal form (2), and the totality of these vectors is therefore characterized by 
the equations 3, = nf,...,7,= 7°, 9,41 = ++- =n, — 0, where n? is the 
ith coordinate of C£?. In the canonical form, the inhomogeneous hypothesis 
£ € II’, therefore becomes n; = n? (i = 1,..., r). This reduces to the homo- 
geneous case on replacing Y, with Y, — 7°, and it follows from (7) that the 
UMP invariant test has the rejection region 


x, - «Dr 
(17) p » C, 
Y ¥7/(n-s) 


i=s+1 


and that the noncentrality parameter is 4? = D7_,(n, — n?)?/0>. 

In applications it is usually most convenient to apply the transformation 
X; — £? directly to (14) or (15). It follows from (17) that such a transforma- 
tion always leaves the denominator unchanged. This can also be seen 
geometrically, since the transformation is a translation of n-space parallel to 
IIg and therefore leaves the distance Y(X,— Ê)? from X to Ig un- 
changed. The noncentrality parameter can be computed as before by 
replacing X with £ in the transformed numerator (16). 

Some examples of linear hypotheses, all with r — 1, were already dis- 
cussed in Chapter 5. The following treats two of these from the present 
point of view. 


Example 4. Let X,,..., X, be independently, normally distributed with com- 
mon mean p and variance c, and consider the hypothesis H : p = 0. Here Ig is 
the line £, = --- = £,, II, is the origin, and s and r are both equal to 1. From the 


identity 
X X 2 AU > 2 ES X 
(X - n) = X(X, - XY + n(X-py, (x- Y- 
n 
it is seen that Ê, = X, while £, = 0. The test statistic and V? are therefore given by 
nx? np? 


W= — eis 
E(x - xy oy c? 
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Under the hypothesis, the distribution of (n — 1)W is that of the square of a 
variable having Student's r-distribution with n — 1 degrees of freedom. 


Example 5. In the two-sample problem considered in Example 1, the sum of 
squares 


bir£ aes code O 


i=1 i=n+1 
is minimized by 
" m x n X. 
í-x»-Yy , 4$-x?- YS, 
i1 i-n41 "2 


while under the hypothesis 7 — £ = 0 


ape ad nuam 
M A: 


The numerator of the test statistic (15), is therefore 


n 


n (t? = Xj n, (AP - HYP = xe - a] 


The more general hypothesis y — £ = 6 reduces to the previous case on replacing 
X, with X, — 0) for i = n, + 1,..., n, and is therefore rejected when 


n, 


(x2 ERE ax » =| 
s T a 
2C. 


E(x xo) + E Qü-39)|mm-2 


i-l ien *l 


H : ?. Under the 
The noncentrality parameter is ? = (n — $ — 6) /(/n, + 1/n)e*. Un 
hypothesis, the Bes root of the test statistic has the /-distribution with mj + n; — 2 


degrees of freedom. 

Explicit formulae for the £, and €; can be obtained by introducing à 
Coordinate system into the parameter space. Suppose in such a system, IIo 
is defined by the equations 


$5 
= Y a;b pep m 


jm 
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or, in matrix notation, 


(18) EsAB, 


nxl  nXxssxl 


where Ais known and of rank s, and f,..., B, are unknown parameters. If 
A: are the least-squares estimators minimizing L,( X, — Y;2,Bj)^, it 
is seen by differentiation that the B, are the solutions of the equations 


AAB = A'X 
and hence are given by 
Ê= (44) ux. 


(That A'A is nonsingular is shown in Lemma 1 of Chapter 8.) Thus, we 
obtain 


£-A(44) wx. 


Since £ = £( X) is the projection of X into the space IIg spanned by the s 
columns of A, the formula Ê= A(4’A)~14’X shows that P = A( A'A) 14’ 
has the property claimed for it in Example 3 of Chapter 6, that for any X in 
R", PX is the projection of X into Ig. 


3. TESTS OF HOMOGENEITY 


The UMP invariant test obtained in the preceding section for testing the 
equality of the means of two normal distributions with common variance is 
also UMP unbiased (Section 3 of Chapter 5). However, when a number of 
populations greater than 2 is to be tested for homogeneity of means, a UMP 
unbiased test no longer exists, so that invariance considerations lead to a 
new result. Let X, (j =1,..., n; i=1,...,s) be independently distrib- 
uted as N(p;, 07), and consider the hypothesis 


H: =- =p, 


This arises, for example, in the comparison of a number of different 
treatments, processes, varieties, or locations, when one wishes to test whether 
these differences have any effect on the outcome X. It may arise more 
generally in any situation involving a one-way classification of the outcomes, 
that is, in which the outcomes are classified according to a single factor. 
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The hypothesis H is a linear hypothesis with r = s — 1, with IIg given 
by the equations £;; = €,, for j, k 2 1,..., n, i — 1,..., s and with II, the 
line on which all n = En, coordinates £;, are equal. We have 


DE = ui) = LE (xy B mn)? E En (X= a)” 


with X;, = L7_,X;,/n,, and hence £ = X;.. Also, 


‘j=l 


ELO #) = LEK, - XJ * n( uy 
with X.= DDX,,/n, so that Ê, = X... Using the form (15) of W'*, the test 
therefore becomes 
En (X X As - 1) 
YX(X,- XJ'/(n - 5) 


(19) w* 
The noncentrality parameter is 


y? 


my Yn; d n) 


with 


Yn, 


KERI 


The sum of squares in both numerator and denominator of (19) admits 
three interpretations, which are closely related: (i) as the two components in 
the decomposition of the total variation 


YX(x,- X) - EEX- X)! * Enl XJ. 


Of which the first represents the variation within, and the second the 
variation between populations; (ii) as a basis, through the test (19), for 
comparing these two sources of variation; (iii) as estimates of their expected 
values, (n — s)o? and (s— 1)o? + Eni(p; — ui) (Problem 13). This 
breakdown of the total variation, together with the various interpretations 
of the components, is an example of an analysis of variance,* which will be 
applied to more complex problems in the succeeding sections. 


*For conditions under which such a breakdown is possible, see Albert (1976). 
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We shall now digress for a moment from the linear hypothesis scheme to 
consider the hypothesis of equality of variances when the variables X,, are 
distributed as N(p; 02), i= 1,...,s. A UMP unbiased test of this hy- 
pothesis was obtained in Chapter 5, Section 3, for the case s = 2, but does 
not exist for s > 2 (see, for example, Problem 6 of Chapter 4). Unfor- 
tunately, neither is there available for this problem a group for which there 
exists a UMP invariant test. To obtain a test, we shall now give a 
large-sample approximation, which for sufficiently large n essentially re- 
duces the problem to that of testing the equality of s means. 

It is convenient first to reduce the observations to the set of sufficient 
statistics X= X,X,;/n, and S? = E(X- X), i= L...,s. The hy- 
pothesis 

H:o,7 = =o 
remains invariant under the transformations X/,= X;; + c;, which in the 
space of sufficient statistics induce the transformations S? = S2, X/.=X;. 
+c, A set of maximal invariants under this group are Sj,..., S2. Each 
statistic S2 is the sum of squares of n; — 1 independent normal variables 
with zero mean and variance o2, and it follows from the central limit 
theorem that for large n; 


is approximately distributed as N(0, 20/). This approximation is inconveni- 
ent for the present purpose, since the unknown parameters o, enter not only 
into the mean but also the variance of the limiting distribution. 

The difficulty can be avoided through the use of a suitable variance- 
stabilizing transforrnation. Such transformations can be obtained with the 
help of Theorem 5 of Chapter 5, which shows that if /n(T, — 0) is 
asymptotically normal with variance 77(@), then Vn[f(T,) — /(0)] is 
asymptotically normal with variance 7?(0)[f/(@)]?. Thus f is variance- 
stabilizing [i.e., the distribution of f(T,) has approximately constant vari- 
ance] if f (8) is proportional to 1/7(0). 

This applies to the present case with n —,— 1, T, = S2/(n; — 1), 
0 = o?, and 7? = 267, and leads to the transformation f(@) = log @ for 
which the derivative is proportional .to 1/0. The limiting distribution of 
jn; — 1 {logl S2/(n; — 1)] - logo?) is the normal distribution with zero 
mean and variance 2, so that for large n, the variable Z, = logi S2/(n; — 91 
has the approximate distribution N(f, a2) with ¢, = log 07, a; = 2/ 
(sed ge 
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The problem is now reduced to that of testing the equality of means of s 
independent variables Z, distributed as N($;, a?) where the a, are known. 
In the particular case that the n; are equal, the variances a2 are equal and 
the asymptotic problem is a simpler version (in that the variance is known) 
of the problem considered at the beginning of the section. The hypothesis 
{= or = §, is invariant under addition of a common constant to each of 
the Z’s and under orthogonal transformations of the hyperplanes which are 
perpendicular to the line Z, — --+ = Z,. The UMP invariant rejection 
region is then 


X(Zz si zy "od 


where a? is the common variance of the Z, and where C is determined by 
i} 

(20) [ xà) 9 - a. 
c 


In the more general case of unequal a,, the problem reduces to a linear 
hypothesis with known variance through the transformation Z/ = Z,/a,, 
and the UMP invariant test under a suitable group of linear transformations 
rejects when 


2 2 2\2 
1 YZ/ai 2 (Xz/ai) 
e) Eaa ua] Aal Da) 
(see Problem 14), where C is again determined by (20). This rejection 
region, which is UMP invariant for testing {, = =- =$, in the limiting 
distribution, can then be said to have this property asymptotically for 
testing the original hypothesis H:0,= 796, 


When applying the principle of invariance, it is important to make sure 
that the underlying symmetry assumptions really are satisfied. In the prob- 
lem of testing the equality of a number of normal means jn pb, for 
example, all parameter points, which have the same value of y = n 47 
u )?/6?, are identified under the principle of invariance. This is appropriate 
only when these alternatives can be considered as being equidistant from the 


hypothesis. In particular, it should then be immaterial whether the given 


value of 4? is built up by a number of small contributions or a single large 


one. Situations where instead the main emphasis is on the detection of large 
and the test 


individual deviations do not possess the required symmetry, 
based on (19) need no longer be optumum. 
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The robustness properties against nonnormality of the /-test, and the 
nonrobustness of the F-test for variances, found in Chapter 5, Section 4 for 
the two-sample problem, carry over to the comparison of more than two 
means or variances. Specifically, the size and power of the F-test (19) of 
H:p, = s — p, is robust for large n; if the Xj; (j=1,..., n;) are 
samples from distributions F(x — p;) where F is an arbitrary distribution 
with finite variance. [A discussion of the corresponding permutation test 
with references to the literature can be found for example in Robinson 
(1983). For an elementary treatment see Edgington (1980).] On the other 
hand, the test for equality of variances described above (or Bartlett’s test, 
which is the classical test for this problem) is highly sensitive to the 
assumption of normality, and therefore is rarely appropriate. More robust 
tests for this latter hypothesis are reviewed in Conover, Johnson, and 
Johnson (1981). 

That the size of the test (19) is robust against nonnormality follows from 
the fact that if the X;,, j=1,..., n; are independent samples from 
F(x — y), then under H: p = --- =p, 


(i) the distribution of the numerator of W *, multiplied by (s — 1)/ a?, 
tends to the x?  , distribution provided n,/n — p; > 0 for all i and 
(i) the denominator of W* tends in probability to o°. 


To see (i), assume without loss of generality that p; = ::: =H, = 0. 
Then the variables "m X,. are independent, each with a distribution which 
by the central limit theorem tends to N(0, o?) as n, > oo for any F with 
finite variance. It follows (see Section 5.1, Theorem 7 of TPE) that for any 
function A, the limit distribution of h(n, X,.,.--,n,X,.) is the distribu- 
tion of h(U,,...,U,) where U,,...,U, are independent N(0, o?), provided 


{(u,,...,u,): A(u,-..,u,) =c} 


has Lebesgue measure 0 for any c. Suppose that n,/n = p; as ny... "s 
tend to infinity. This condition is satisfied for 


h(/ni X... n,X,) = Dn (X7 X. 


and the limit distribution of the numerator of W* is (for all F with finite 
variance) what it is when F is normal, namely o? times x? ,. A slight 
modification shows the result to remain true if n,/n > p;. 


tFor a discussion of this test, see for example Cyr and Manoukian (1982) and Glaser 
(1982). 
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Part (ii) is a special case of the following more general result: Let 
X,,..., X, be independently distributed, X, according to F(x, — p;) with 
E(X,) = p; and Var(X;) = o? < co, and suppose that for each n the vector 
(iis ---» En) is known to lie in an s-dimensional space IIg, with s fixed. 
Then the denominator D of (14) tends to c? in probability as n — oo. 

This can be seen from the canonical form (7) of W*, in which 


n n s 
p> $e T Ee 

N—S i=s+1 n—s[ni-i n=s i=] 
and the fact that ZY2/n = EX?/n. Since E(Y,) = 0 for i =s + 1... m 
assume, without loss of generality for the distribution of X7, ,Y?, that 
E(X,) = E(¥,) = 0 for all i. Then by the law of large numbers EX?/n 
tends in probability to E(X?) = 07. On the other hand, we shall now show 
that the second term on the right side of D tends in probability to zero. The 

result then follows. 

To see this, it is enough to show that each of Y2,..., Y? is bounded in 
probability. Now Y, = Ec Xj, where the vectors (c(,..., cf”) are or- 


thogonal and of length 1. Therefore, by the Chebyshev inequality 
1 puo? 
Plaza) < E(P) = a 


and this completes the proof. 

‘Another robustness aspect of the s-sample F-test concerns the assump- 
tion of a common variance. Here the situation is even worse than in the 
two-sample case. If the X;; are independently distributed as N( pj, 67) and 
if s > 2, the size of the F-test (19) of H:p, = °°: = p, is not asymptoti- 
cally robust as n; > 00, n;/En; > Pr regardless of the values of the p, 
[Scheffé (1959). More appropriate tests for this generalized Behrens-Fisher 
problem have been proposed by Welch (1951), James (1951), and Brown 
and Forsythe (19742), and are further discussed by Clinch and Kesselman 
(1982). The corresponding robustness problem for more general linear 
hypotheses is treated by James (1954) and Johansen (1980); see also 
Rothenberg (1984). Á 

The linear model F-test—as was seen to be the case for the t-test—is 
highly nonrobust against dependence of the observations. Tests of the 
hypothesis that the covariance matrix is proportional to the identity against 
various specified forms of dependence are considered in King and Hillier 
(1985). [ ; 

The test (19), although its level and power are asymptotically indepen- 


dent of the distribution F, tends to be inefficient if F has heavier tails than 
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the normal distribution. More efficient tests are obtained by generalizing the 
considerations of Sections 8 and 9 of Chapter 6. Suppose the X,, are 
samples of size n; from continuous distributions F, (i = 1,..., s) and that 
we wish to test H: F, = -+:, = F, Invariance, by the argument of Chapter 
6, Section 8, then reduces the data to the ranks R, j of the X;; in the 
combined sample of m — Xn, observations. A natural analogue of the 
two-sample Wilcoxon test is the Kruskal-Wallis test, which rejects H when 
Xn((R,— R..)* is too large. For the shift model F,(y) = F(y — p,), the 
asymptotic efficiency of this test relative to (19) is the same as that of the 
Wilcoxon to the t-test in the case s = 2. The theory of this and related rank 
tests is developed in books on nonparametric statistics such as Hájek 
and Sidák (1967) Lehmann (1975), Randles and Wolfe (1979), and 
Hettmansperger (1984). 

Unfortunately, such rank tests are available only for the very simplest 
linear models. An alternative approach capable of achieving similar 
efficiencies for much wider classes of linear models can be obtained through 
large-sample theory. It replaces the least-squares estimators by estimators 
with better efficiency properties for nonnormal distributions and obtains an 
asymptotically valid significance level through *'Studentization",* that is, by 
dividing the statistic by a suitable estimator of its standard deviation. 
Different ways of implementing such a program are reviewed, for example, 
by Draper (1981, 1983), McKean and Schrader (1982), and Ronchetti 
Can [For a simple alternative of this kind to Student’s f-test, see Prescott 
1975).] 

Sometimes, it is of interest to test the hypothesis H: p; = + = ps 
considered at the beginning of the section, against only the ordered alterna- 
tives uj S -+> <p, rather than against the general alternatives of any 
inequalities among the p’s. Then the F-test (19) is no longer reasonable; 
more powerful alternative tests for this and other problems involving 
ordered alternatives are discussed in Barlow et al. (1972). 


4. MULTIPLE COMPARISONS 


Testing equality of a number of means as a simple choice between accep- 
tance and rejection usually leaves many questions unanswered. In particu- 
lar, when the hypothesis is rejected one would like to obtain more detailed 


. "This term (after Student, the pseudonym of W. S. Gosset) is a misnomer. The procedure of 
dividing the sample mean X by its estimated standard deviation and referring the resulting 
statistic to the standard normal distribution (without regard to the distribution of the X's) was 
used already by Laplace. Student's contribution consisted in pointing out that if the X's are 
normal, the approximate normal distribution of the r-statistic can be replaced by its exact 
distribution— Student's z. 
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information about the relative positions of the means. In order to determine 
just where the differences in the p’s occur, one may want to begin by testing 
the hypothesis H,: p, = -:- = pp as before, with the F-test (19). If this 
test accepts, the means are judged to exhibit no significant differences, the 
set (I, ..., I4, ) is declared homogeneous, and the procedure terminates. If 
H, is rejected, a search for the source of the differences can be initiated by 
proceeding to a second stage, which consists in testing the s hypotheses 


Hy m SUE p E mu, 


by means of the appropriate F-test for each. This requires the obvious 
modification of the numerator of (19), while the denominator is being 
retained at all the steps. This is justified by the assumption of a common 
variance o? of which the denominator is an estimate. For any hypothesis 
that is accepted, the associated set of means and all its subsets are judged 
not to have shown any significant differences and are not tested further. For 
any rejected hypothesis the s — 1 subsets of size s — 2 are tested [except 
those that are subsets of an (s — l)set whose homogeneity has been 
accepted], and the procedure is continued in this way until nothing is left to 
be tested. 

It is clear from this description that a particular set of p’s is declared 
heterogeneous if and only if the hypothesis of homogeneity is rejected for it 
and all sets containing it. 

Instead of the F-tests, other tests of homogeneity could be used at the 
various stages. When the sample sizes n; = n are equal, as we shall assume 
throughout the remainder of this section, the most common alternative is 
based on the Studentized range statistic 


max|X,.— Xl 


where the maximum is taken over all pairs (i, j) within the set being tested. 
We shall here restrict attention to procedures where the test statistics are 


either F or Studentized range, not necessarily the same at all stages. 


T description of the procedure, once the test statistics have 
uc huc. the critical values which they must 


been chosen, it is necessary to specify criti ) 
exceed for rejection, or equivalently, the significance levels at which the 
various tests are to be performed. Suppose all tests at a given stage are 
performed at the same level, and denote this level by æ, when the equality 


of k means is being tested, and the associated critical values by Cx, 


ko. 
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Before discussing the best choice of a’s let us consider some specific 
methods that have been proposed in the literature. Additional properties 
and uses of some of these will be mentioned at the end of the section. 


(i) Tukey’s T-method. This procedure employs the Studentized range 
test at each stage with a common critical value C, = C for all k. The 
method has an unusual feature which makes it particularly simple to apply. 
In general, in order to determine whether a particular subset S, of means 
should be called nonhomogeneous, it is necessary to proceed stagewise since 
the homogeneity of S, itself is not tested unless homogeneity has been 
rejected for all sets containing Sy. However, with Tukey's T-method it is 
only necessary to test Sp itself. If the Studentized range of Sọ exceeds C, so 
will that of any set containing Sy, and S, is declared nonhomogeneous. In 
the contrary case, homogeneity of S, is accepted. The two facts which 
jointly eliminate the need for a stagewise procedure in this case are (a) that 
the range, and hence the Studentized range, of Sọ cannot exceed that of any 
set S containing Sọ, and (b) the constancy of the critical value. The next 
method applies this idea toa procedure based on F-tests. 

(i) Gabriel’s simultaneous test procedure. F-statistics do not have 
property (a) above. However, this property is possessed by the statistics »F, 
where v is the number of numerator degrees of freedom (Problem 16). 
Hence a procedure based on F-statistics with critical values C, = C/(k — 1) 
satisfies both (a) and (b), since k — 1 is the number of numerator degrees of 
freedom when k means are being tested, that is, at the (s — k + 1)st stage. 
This procedure, which in this form was proposed by Gabriel (1964), permits 
the testing of many additional hypotheses and when these are included 
becomes Scheffé’s S-method, which will be discussed in Sections 9 and 10. 

Gii) Fisher’s least-significant-difference method employs an F-test at the 
first stage, and Studentized range tests with a common critical value 
C,-1 = ++: = C, at all succeeding stages. The constants C, and C, are 
related by the fact that the first stage F-test and the pairwise t-test of the 
last stage have the same level. 


The usual descriptions of (iii) and (i) consider only the first and last stage 
of these procedures, and omit the conclusions which can be drawn from the 
intermediate stages. 

_ Several classes of procedures have been defined by prescribing the 
significance levels a,, which can then be applied to the chosen test statistic 
at each stage. Examples are: 


(iv) The Newman-Keuls levels: 


a, = a. 
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(v) The Duncan levels: 


a,=1- yee 


(vi) The Tukey levels: 


on 1- y^, hii su 
KE Eye em sii, s. 


In both (v) and (vi), y = 1 — az. 

Most of the above methods and some others are reviewed and their 
justification discussed by Spjetvoll (1974); comparisons of different methods 
are provided, for example, by Einot and Gabriel (1975). 

Let us now consider the choice of the levels a, more systematically. In 
generalizing the usual significance level a for a single test, it is desirable to 
control some overall measure of the extent to which a procedure leads to 
false rejections. One such measure is the maximum probability a of at least 
one false rejection, that is, of rejecting homogeneity of at least one set of p’s 
which is in fact homogeneous. The probability of at least one false rejection 
for a given (p,,..., Ms) will be denoted by a(J;,..., Hs), so that a = 
sup a(p,, . . . fs), Where the supremum is taken over all s-tuples (py, -<-> Hs). 

In order to study the best choice of a5, .. ., 4; subject to 


(23) a, € ağ 


for a given level ağ, let us simplify the problem by assuming c? to be 
known, say o? — 1. Then the F-tests (19) are replaced by the x^-tests with 
rejection region Xn,(X,.— X.) > C, and the Studentized range tests are 
replaced by the range tests which reject when the range of the subgroup 
being tested is too large. 

Theorem 1. Suppose that at each stage either a x?- or a range test is used 
(not necessarily the same at all stages) and that the p’s fall into r distinct 
groups of sizes v, .. ., 0, (LY; = $), Say 


(24) M m c E fee. S t Pap te 
where (i,,...,i,) is a permutation of (1,..., 5). Then 


(25) sup (p, ... Hs) = 1— I - an) 


where a, = 0 and the supremum is taken over all (p, --- p,) satisfying (24). 
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Proof. Since false rejection can occur only when at least one of the 
hypotheses 


Q6) Hia ete = mp HES Thurs 


is rejected, 
a(py,.--,H;) < P (rejecting at least one H/) 


= 1 — P (accepting all the H7) 


r 


21- [I(1- o,). 


i=l 


Here the last equality follows from the fact that the test statistics for testing 
the hypotheses Hý, ..., H/ are independent. 

To see that the upper bound is sharp, let the distances between the 
different groups of means (24) all tend to infinity. Then the probability of 
accepting homogeneity of any set containing (H;,.... i, } as a proper 
subset, and therefore not reaching the stage at which Hj is tested, tends to 
zero. The same is true for Hj,..., H/, and hence a(ut,, . .., Hs) tends to the 
right side of (25). 

It is interesting to note that sup a(J,, ..., p,) depends only on a5, ... &s 
and not on whether x?- or range statistics are used at the various stages. In 
fact, Theorem 1 remains true for many other statistics (Problem 17). 

It follows from Theorem 1 that a procedure with levels (ay, Sa 
satisfies (23) if and only if 


r 
(27) E e a forall hoins: 0) with Lus. 
i-i 


To see how to choose a5, ..., a, subject to (23) or (27), let us say that 
(a5, ..., &) is inadmissible if there exists another set of levels (a5, ..., 65) 
satisfying (27) and such that 


(28) a,<a’ forall i, with strict inequality for some i. 


These inequalities imply that the procedure with the levels a; has uniformly 
better chance of detecting existing inhomogeneities than the procedure 
based on the a,. The definition is thus in the spirit of a-admissibility 
discussed in Chapter 6, Section 7. 
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Lemma 1. Under the assumptions of Theorem 1, necessary conditions for 
(a5, ..., a) to be admissible are 

(i) a< +++ <a, and 

Gi) a, = a, = a§. 

Proof. (i): Suppose to the contrary that there exists k such that a, ) < 
ap, and consider the procedure in which aj = a, for i + k + 1 and ak, = 
a,,. To show that a < ağ, we need only show that I(1 — a5) = 1 — af for 
all (0,, ..., 0,). If none of the v’s is equal to k + 1, then aj, = a, for all i, 
and the result follows. Otherwise replace each v that is equal to k + 1 by 
two v’s—one equal to k and one equal to 1—and denote the resulting set of 
v’s by «,..., Wp Then 


r r 
TI 72) - II 7a.) 217 ot 
i=l i=l 

(ii): The left side of (27) involves a, if and only if r = 1, v, = s. Thus the 
only restriction on a, is a, < af, and the only admissible choice is a, = ağ. 
The argument for a,_, is analogous. 

Part (ii) of this lemma shows that procedures (i) and (ii) are inadmissible 
since in both «,_, < a. The same argument shows Duncan’s set of levels to 
be inadmissible. [However, choices (i), (ii), and (v) can be justified from 
other points of view; see for example Spjetvoll (1974) and comment 5 at the 
end of the section.] It also follows from the lemma that for s — 3 there is a 
unique best choice of levels, namely a; = 03 = aj. 

Having fixed ay = a, = a,., = aĝ, how should we choose the remaining 
a’s? In order to have a reasonable chance of detecting existing inhomogene- 
ities for all patterns, we should like to have none of the a's too small. In 
view of part (i) of Lemma 1, this aim is perhaps best achieved by maximiz- 
ing a. x 

Lemma 2. Under the assumptions of Theorem 1, the maximum value of 
a, subject to (23) is 


-1 
(29) a T— (l= ag)?! 


where [A] denotes the largest integer < A. 


Proof. Instead of fixing ag at aĝ and maximizing a, it is more 


convenient instead to fix a», at, say 4 and then to minimize a. The lemma 


will be proved by showing that the resulting minimum value of ao is 


ag 21-4 - 2)". 
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Suppose first that s is even. Since a, is fixed at a, it follows from Theorem 
1 that the right side of (25) can be made arbitrarily close to aj. This is seen 
by letting v, = ::: == 2. When s is odd, the same argument applies 
if we put an additional v equal to 1. 

Lemmas 1 and 2 show that any procedure with a, = a,, and hence 
Fisher's least-significant-difference procedure and the Newman-Keuls choice 
of levels, is admissible for s = 3 but inadmissible for s > 4. The second of 
these statements is seen from the fact that a) < af implies a; < 1 — (1 — 
ağ)!" < ağ when s > 4. The choice a, = a, thus violates Lemma 1(ii). 

Once a, has been fixed at the value given by Lemma 2, it turns out that 
subject to (23) there exists a unique optimal choice of the remaining a's 
when s is odd, and a narrow range of choices when s is even. 


Theorem 2. When s is odd, then a3,...,a, are maximized, subject to 
(23) and (29), by 


(30) af21-(-2)"7, 
and these values can be attained simultaneously. 


Proof. If we put y, — 1 — a; and y = y, then by (27) and (29) any 
procedure satisfying the conditions of the theorem must satisfy 


Iiv, > vA = ye», 


Let i be odd, and consider any configuration in which v, = i and all the 
remaining v’s are equal to 2. Then 


yY8792 > YEDA, 
and hence 
(31) ‘yzy =l- a*. 


An analogous argument proves (31) for even i. 

Consider now the procedure defined by y; — y*. This clearly satisfies 
(29), and it only remains to show that it also satisfies (23) or equivalently 
(27), and hence that 


Tl) > yep? 


or that 


5 [U; XE 
PEE Tin 
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Now L[v,/2] = (s — b)/2, where b is the number of odd v's (including 
ones). Since s is odd, b > 1, and this completes the proof. 

Note that the levels (30) are close to the Tukey levels (vi), which are 
admissible but do not satisfy (29). 

When s is even, a uniformly best choice is not available. In this case, the 
Tukey levels (vi) satisfy (29), are admissible, and constitute a reasonable 
choice. [See Lehmann and Shaffer (1979).] 

Even in the simplified version with known variance the multiple testing 
problem considered in the present section is clearly much more difficult than 
the testing of a single hypothesis; the solution presented above still ignores 
many important aspects of the problem. 


1. Choice of test statistic. The most obvious feature that has not been 
dealt with is the choice of test statistics. Unfortunately it does not appear 
that the invariance considerations which were so helpful in the case of a 
single hypothesis play a similar role here. 

2. Order relation of significant means. Whenever two means X;., Xj. 
are judged to differ, we should like to state not only that p; * Hj, but that if 
X, « X,, then also p; < pj- Such additional statements introduce the possi- 
bility Of additional errors (stating H; < Fj when in fact p; > p), and it is 
not obvious that when these are included, the probability of at least one 
error is still bounded by a£. [This problem of directional errors has been 
solved in a simpler situation in Shaffer (1980).] 

3. Nominal versus true levels. The levels a2,.--1 € sometimes called 
nominal levels, are the levels at which the hypotheses p; = Mj, Hi = M; — 
Las... are tested. They are however not the true probabilities of falsely 
rejecting the homogeneity of these sets, but only the upper bounds of these 
probabilities with respect to variation of the remaining p's. The true 
probabilities tend to be much smaller (particularly when s is large), since 
they take into account that homogeneity of a set So is rejected only if it is 
also rejected for all sets S containing So. 

4. Interpretability. The totality of acceptance and rejection statements 
resulting from a multiple comparison procedure typically does not lead toa 
simple pattern of means. This is illustrated by the possibility that the 
hypothesis of homogeneity is rejected for a set S but for none of its subsets. 
As another example, consider the case s — 3, where it may happen that the 
hypotheses p; = p; and ji; = Hx are accepted but p; = Hx 18 rejected. The 
number of such “inconsistencies” and the corresponding difficulty of inter- 
preting the results may be formidable. Measures of the complexity of the 
totality of statements as a third criterion (besides level and power) are 


discussed by Shaffer (1981). 
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5. Procedures (i) and (ii) can be inverted to provide simultaneous 
confidence intervals for all differences p; — p; The T-method (discussed in 
Problems 65-68) was designed to give simultaneous intervals for all dif- 
ferences p, — p; it can be extended to cover also all contrasts in the p’s, 
that is, all linear functions Ecu; with Ec; = 0, but against more complex 
contrasts the intervals tend to be longer than those of Scheffé’s S-method, 
which was intended for the simultaneous consideration of all contrasts. [For 
a comparison of the two methods, see for example Scheffé (1959, Section 
3.7) and Arnold (1981, Chapter 12).] It is a disadvantage of the remaining 
(truly stagewise) procedures of this section that they do not permit such an 
inversion. 

6. To control the rate of false rejections, we have restricted attention to 
procedures controlling the probability of at least one error. This is some- 
times called the error rate per experiment, since it counts any experiment as 
faulty in which even one false rejection occurs. Instead, one might wish to 
control the expected proportion or number of false rejections. An optimality 
theory based on the latter criterion is given in Spjetvoll (1972). 

7. The optimal choice of the a discussed in this section can be further 
improved, at the cost of considerable additional complication, by permitting 
the a's to depend on the outcomes of the other tests. This possibility is 
discussed, for example, in Marcus, Peritz, and Gabriel (1976); see also Holm 
- (1979) and Shaffer (1984). 

8. If the variance o? is unknown, the dependence introduced by the 
common denominator S when X, is replaced by X,/S invalidates Theorems 
1 and 2, and no analogous results are available in this case. 


5. TWO-WAY LAYOUT: ONE OBSERVATION PER CELL 


The hypothesis of equality of several means arises when a number of 
different treatments, procedures, varieties, or manifestations of some other 
factors are to be compared. Frequently one is interested in studying the 
effects of more than one factor, or the effects of one factor as certain other 
conditions of the experiment vary, which then play the role of additional 
factors. In the present section we shall consider the case that the number of 
factors affecting the outcomes of the experiment is two. 

Suppose that one observation is obtained at each of a number of levels of 
these factors, and denote by X; (i=1,...,a; j=1,...,b) the value 
observed when the first factor is at the ith and the second at the jth level. It 
is assumed that the X;, are independently normally distributed with con- 
stant variance o°, and for the moment also that the two factors act 
independently (they are then said to be additive), so that £,, is of the form 
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a; + Bj. Putting p = a+ B/ and a; — a; — a^, B, = Bj — B: , this can be 
written as 


(32) fíy-ptat Bj, Ya, m xB = 0, 


where the a's and f’s (the main effects of A and B) and p are uniquely 
determined by (32) as* 


(33) a= 6.5 dBee5t-6. web 
Consider the hypothesis 
(34) H:a,= 2,70 


that the first factor has no effect on the outcome being observed. This arises 
in two quite different contexts. The factor of interest, corresponding say to a 
number of treatments, may be f, while a corresponds to a classification 
according to, for example, the site on which the observations are obtained 
(farm, laboratory, city, etc.). The hypothesis then represents the possibility 
that this subsidiary classification has no effect on the experiment so that it 
need not be controlled. Alternatively, a may be the (or a) factor of primary 
interest. In this case, the formulation of the problem as one of hypothesis 
testing would usually be an oversimplification, since in case of rejection of 
H, one would require estimates of the a’s or at least a grouping according to 
high and low values. 

The hypothesis H is a linear hypothesis with r = a — 1, s = 1-4 (a-1) 
+(b-1)=a+b-1, and n-s= (a — Y(b — 1) The least-squares 
estimates of the parameters under Q can be obtained from the identity 


XX(X - £f = DECK - B) 
= EN x= 0x) O a;) 
+(X; - X— B) + (Xe "mi 
= (aya X2 Xm XY + b (X. Xm e) 
*aX(X.;- X.— B)! + ab(X.— ny, 


*The replacing of a subscript by a dot indicates that the variable has been averaged with 


respect to that subscript. 
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which is valid because in the expansion of the third sum of squares the 
cross-product terms vanish. It follows that 


&=X,-X., B-Xj-X. h-X. 
and that 
YX(X; a &)y ES DEE Ta X= X.j + Hih 


Under the p. H we still have Ê = X.,- X.. and fi= X.., and 
hence £;; — £j; = Xi. X... The best invariant test therefore rejects when 


Q5) w*- bY(X,— X.)/(a - 1) n 


YXX(X,-X.—Xj* X. /(a — 1)(b = 1) 


The noncentrality parameter, on which the power of the test depends, is 
given by 


Elt)  bEe 


o o? 


(36) y 


This problem provides another example of an analysis of variance. The 
total variation can be broken into three components, 


YX(X,- X) -5E(QL- XJ + aE(x.- X-) 


XE, 7 X X, X). 


Of these, the first contains the variation due to the a's, the second that due 
to the B's. The last component, in the canonical form of Section 1, is equal 
to E^... Y. It is therefore the sum of squares of those variables whose 
means are zero even under Q.. Since this residual part of the variation, which 
on division by n — s is an estimate of o?, cannot be put down to any effects 
such as the a’s or B's, it is frequently labeled “error,” as an indication that 
it is due solely to the randomness of the observations, not to any differences 
of the means. Actually, the breakdown is not quite as sharp as is suggested 
by the above description. Any component such as that attributed to the «’s 
always also contains some “error,” as is seen for example from its expecta- 
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tion, which is 
EY(X,— X. = (a - 1)o? + bYa?. 


Instead of testing whether a certain factor has any effect, one may wish to 
estimate the size of the effect at the various levels of the factor. Other 
parameters, which it is sometimes interesting to estimate, are the average 
outcomes (for example yields) £,.,..., a. when the factor is at the various 
levels. If 6, = p + a; = £;., confidence sets for (0,,..-,9,) are obtained by 
considering the hypotheses H(8°): 6, = 0? (i = 1,..., a). For testing = 
ox 6, = 0, the least-squares estimates of the £;; are £; -XGX,-X. 
and £,,— X.; — X... The denominator sum of squares is therefore LOX, j 
P ior cpi X.)* as before, while the numerator sum of squares is 


i* 


ELl an i) = by x. 


The general hypothesis reduces to this special case on replacing X; with the 
variable X,, — 0°. Since s c a b - 1 and r = a, the hypothesis H(0^) is 
rejected when 


bY(X.— 69 /a un 
YX(Xx,-X-X,* X.) /(a — 1)(b - 1) 


The associated confidence sets for (6, ..- 6,) are the spheres 


2 aCY (X, Xi Xi + xJ 
Do; EL 345 < aS CS ae 


When considering confidence sets for the effects a, ..., €a one must take 
account of the fact that the a's are not independent. Since they add up to 
zero, it would be enough to restrict attention to a4, . +, 6-1 However, an 
easier and more symmetric solution is found by retaining all the a's. The 

jecti i :a,;— aj j=1,..., a (with La; = 
rejection region of H: a; a? for i=1,...,4 ( th Xa — 0) is obtained 
from (35) by letting X/, = X; — a, and hence is given by 
nrg GbE Qu A Xu x 
bY(X— X.— aj) = BEET d 


The associated confidence set consists of the totality of points (a5, ..-; aa) 
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satisfying Xa, = 0 and 


CXX(X,- X— Xy + X-) 
Ela- (%- XJ < M een Be) 


In the space of (ay. -+s 4), this inequality defines a sphere whose center 
(X,.— X..,..., X,— X.) lies on the hyperplane Xa; = 0. The confidence 
sets for the a's therefore consist of the interior and surface of the great 
hyperspheres obtained by cutting the a-dimensional spheres with the hyper- 
plane Xa, — 0. 

In both this and the previous case, the usual method shows the class of 
confidence sets to be invariant under the appropriate group of linear 
transformations, and the sets are therefore uniformly most accurate in- 
variant. 

A rank test of (34) analogous to the Kruskal-Wallis test for the one-way 
layout is Friedman's test, obtained by ranking the s observations X; j, -- -> Xj 
separately from 1 to s at each level j of the second factor. If these ranks are 
denoted by Rj;,---, Rsjs Friedman's test rejects for large values of Z(R;.— 
R.)?. Unless s is large, this test suffers from the fact that comparisons are 
restricted to observations at the same level of factor 2. The test can be 
improved by “aligning” the observations from different levels, for example, 
by subtracting from each observation at the jth level its mean X., for that 
level, and then ranking the aligned observations from 1 to ab. For a 
discussion of these tests and their efficiency see Lehmann (1975, Chapter 6), 
and for an extension to tests of (34) in the model (32) when there are several 
observations per cell, Mack and Skillings (1980). Further discussion is 
provided by Hettmansperger (1984). 

That in the experiment described at the beginning of the section there is 
only one observation per cell, and that as a consequence hypotheses about 
the a’s and fs cannot be tested without some restrictions on the means [377 
does not of course justify the assumption of additivity. Rather, the other 
way around, the experiment should not be performed with just one observa- 
tion per cell unless the factors can safely be assumed to be additive. Faced 
with such an experiment without prior assurance that the assumption holds, 
one should test the hypothesis of additivity. A number of tests for this 
purpose are discussed, for example, in Hegemann and Johnson (1976) and 
in Marasinghe and Johnson (1981). 


6. TWO-WAY LAYOUT: m OBSERVATIONS PER CELL 


In the preceding section it was assumed that the effects of the two factors & 
and f are independent and hence additive. The factors may, however, 
interact in the sense that the effect of one depends on the level of the other. 
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Thus the effectiveness of a teacher depends for example on the quality or 
the age of the students, and the benefit derived by a crop from various 
amounts of irrigation depends on the type of soil as well as on the variety 
being planted. If the additivity assumption is dropped, the means £;; of X;; 
are no longer given by (32) under €) but are completely arbitrary. More than 
ab observations, one for each combination of levels, are then required, since 
otherwise s — n. We shall here consider only the simple case in which the 
number of observations is the same at each combination of levels. 

Let X; (i= 1,72; p TES mS m) be independent nor- 
mal with common variance o? and mean E( X, jx) = £;. In analogy with the 
previous notation we write 


y= £(E- E) + (657 6) * (Gu 6 Ey t+ £.) 
=p +a t+ B+ Yy 
with Z;a; = EB; = Xj = Ey; = 0. Then a, is the average effect of factor 
1 at level i, averaged over the b levels of factor 2, and a similar interpreta- 
tion holds for the "s. The y's are called interactions, since y,; measures the 
extent to which the joint effect £,, — &.. of factors 1 and 2 at levels i and j 
exceeds the sum (£,— £.) + (£.; — £.) of the individual effects. Consider 


again the hypothesis that the a’s are zero. Then r= a — 1, s = ab, and 
n — s = (m — 1)ab. From the decomposition 


LLL (Xp E) = EEL (Xie Xy) + MLE KH i) 
and 
TE (Kym tp) = EEX Xe X X-— wY 
$bY (Xm Xun a) + aL (Kym X«— B) 


+ab( X...— n» 


it follows that 
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and hence that 
TELTET p &) -— TP z iris 
EXX(&,- £J = mY(x.- x.y. 
The most powerful invariant test therefore rejects when 


mb) (X,.— X..)/(a —1) 


37 ws- cats 
a LEE (Xj - X,)'/(m — 1)ab 


and the noncentrality parameter in the distribution of W* is 


mbY(£.—£.) mba? 
o? g^ rar 


(38) 


Another hypothesis of interest is the hypothesis H’ that the two factors 
are additive,t 


H':y,-0  foralli, j. 


The least-squares estimates of the parameters are easily derived as before, 
and the UMP invariant test is seen to have the rejection region (Problem 22) 


mYY(X,— X.— Xj x.)'/(a- 1)(5 - 1) 


(39) w*- : 
DE. -= X.) /(m - 1)ab 


C. 


Under H’, the statistic W* has the F-distribution with (a — 1)(b — 1) and 
(m — 1)ab degrees of freedom; the noncentrality parameter for any alterna- 
tive set of y's is 


(40) pe mk 


'A test of H^ against certain restricted alternatives has been proposed for the case of one 
observation per cell by Tukey (1949); see Hegemann and Johnson (1976) for further discussion. 
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The decomposition of the total variation into its various components, in 
the present case, is given by 


YYY OQ - Xa) = mb) (X..— X.) + ma (X.,— Xa)? 
*mYYM(X,— Xn X. Po 


+ XXX A A. 


Here the first three terms contain the variation due to the a's, B's and y’s 
respectively, and the last component corresponds to error. The tests for the 
hypotheses that the a's, B's, or y’s are zero, the first and third of which 
have the rejection regions (37) and (39), are then obtained by comparing the 
a, B, or y sum of squares with that for error. 

An analogous decomposition is possible when the y's are assumed a 
priori to be equal to zero. In that case, the third component which 
previously was associated with y represents an additional contribution to 
error, and the breakdown becomes 


EES (Xij = X.) = mbY (X.— X.) + ma (X. X.) 
+ DEE (Xie — Xam X42 X). 


with the last term corresponding to error. The hypothesis H: a, = ++- 
a, = 0 is then rejected when 


mb (X;..- X..)"/(a — 1) 
DEL (Kye Xu Xy X.) gm - a-b*1) 


C 


Suppose now that the assumption of no interaction, under which this test 
was derived, is not justified. The denominator sum of squares then has a 
noncentral x?-distribution instead of a central one; and is therefore sto- 
chastically larger than was assumed (Problem 25). It TOONE that the actual 
rejection probability is less than it would be for Xy; = 0. This shows that 
the probability of an error of the first kind will not exceed the nominal level 
of significance, regardless of the values of the y's. However, the power also 
decreases with increasing EEy2/0 and tends to zero as this ratio tends to 
infinity. [400 aeqne : 

The analysis of variance and the associated tests derived in this section 
for two factors extend in a straightforward manner to a larger number of 
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factors (see for example Problem 26). On the other hand, if the number of 
observations is not the same for each combination of levels (each cell), 
explicit formulae for the least-squares estimators may no longer be avail- 
able, but there is no difficulty in computing these estimators and the 
associated UMP invariant tests numerically. However, in applications it is 
then not always clear how to define main effects, interactions, and other 
parameters of interest, and hence what hypothesis to test. These issues are 
discussed, for example, in Hocking and Speed (1975) and Speed, Hocking, 
and Hackney (1978). See also TPE, Chapter 3, Example 4.4, and Arnold 
(1981, Section 7.4). 

Of great importance are arrangements in which only certain combina- 
tions of levels occur, since they permit reducing the size of the experiment. 
Thus for example three independent factors, at m levels each, can be 
analyzed with only m? observations, instead of the m? required if 1 
observation were taken at each combination of levels, by adopting a 
Latin-square design (Problem 27). 

The class of problems considered here contains as a special case the 
two-sample problem treated in Chapter 5, which concerns a single factor 
with only two levels. The questions discussed in that connection regarding 
possible inhomogeneities of the experimental material and the randomiza- 
tion required to offset it are of equal importance in the present, more 
complex situations. If inhomogeneous material is subdivided into more 
homogeneous groups, this classification can be treated as constituting one or 
more additional factors. The choice of these groups is an important aspect 
in the determination of a suitable experimental design.j A very simple 
example of this is discussed in Problems 49 and 50 of Chapter 5. 

Multiple comparison procedures for two-way (and higher) layouts are 
discussed by Spjetvoll (1974); additional references can be obtained from 
the bibliography of R. G. Miller (1977). 


7. REGRESSION 


Hypotheses specifying one or both of the regression coefficients «, 8 when 
X,,..., X, are independently normally distributed with common variance 
o° and means 


(41) ;7«* t, 


* For a discussion of various designs and the conditions under which they are appropriate 
see, for example, Cox (1958), John (1971), John and Quenouille (1977), and Box, Hunter, and 
Hunter (1978). Optimum properties of certain designs, proved by Wald, Ehrenfeld, Kiefer, and 
others, are discussed by Kiefer (1958, 1980) and Silvey (1980). The role of randomization, 
treated for the two-sample problem in Chapter 5, Section 12, is studied by Kempthorne (1955), 
Wilk and Kempthorne (1955), Scheffé (1959), and others; see, for example, Lorenzen (1984). 
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are essentially linear hypotheses, as was pointed out in Example 2. The 
hypotheses H : a = a, and H;: B = By were treated in Chapter 5, Section 
8, where they were shown to possess UMP unbiased tests. We shall now 
consider H, and H,, as well as the hypothesis H;: a = a, B = Bo, from 
the present point of view. By the general theory of Section 1 the resulting 
tests will be UMP invariant under suitable groups of linear transformations. 
For the first two cases, in which r = 1, this also provides, by the argument 
of Chapter 6, Section 6, an alternative proof of their being UMP unbiased. 

The space IIg is the same for all three hypotheses. It is spanned by the 
vectors (1,...,1) and (t... tn) and has therefore dimension s — 2 unless 
the /, are all equal, which we shall assume not to be the case. The 
least-squares estimates a and f! under Q are obtained by minimizing 
X(X, — a — Bt,)?. For any fixed value of B, this is achieved by the value 
a= X —Bi, for which the sum of squares reduces to X[(X, — X)- 
A(t, — Ż)]?. By minimizing this with respect to B one finds 


panpin 
42 sc EE NO e X = Br 
TI Po esed 
and 


E(x- â- Êu) = Yüosxy-PrG-0 


is the denominator sum of squares for all three hypotheses. The numerator 
of the test statistic (7) for testing the two hypotheses a = 0 and f = 0 is Y 
and for testing a = B = 0 is Y? + Y? | 

For the hypothesis a — 0, the statistic Y, was shown in Example 3 to be 


equal to 
EVI 
fi gee Xt i) 
NU ME it = 
T Er 
Since then 


is equivalent to the hypothesis E(Y1) == 


the hypothesis a = & : 
agy nX(t; - Dx, for which the rejection region (17) is (n — SXYy — 
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*/XL- Y? > Co and hence 


]& — aly L(t, s DPT. 
Woke 


(EQ - à - BuY/(n - 2) 


(43) 


For the hypothesis 8 = 0, Y, was shown to be equal to 


L Xyt = 1) "mera 


E(g-i 
Since then E(Y,) = B L(t, -i Vas the hypothesis B = Bo is equivalent to 
EY) = 18 = Boy X(t; — i)" and the rejection region is 
1B = Boly D(x) i 
(xo irs Bi) /(n 202) 


For testing a = f = 0, it was shown in Example 3 that 


(44) 2.065. 


Y,- B/X(t- i). &-vnX- Vn(à + Bi); 
and the numerator of (7) is therefore 


y2+¥2 n(a+ fi) + BL (1-74) 


are 2 
The more general hypothesis a = a, B = Bo is equivalent to E(Y,) = mi 
E(Y,) = 19, where m? = Boy E(t, — i), 03 Vn (ao + Bof); and the rejec- 
tion region (17) can therefore be written as 


[n(à — ao)?  2ni(à - ao)(B - Bo) + E1? (Ê - Poy |/2 . c 
E(X- â- Buy /(n- 2) 
The associated confidence sets for (a, B) are obtained by reversing this 


inequality and replacing a, and f, by a and f. The resulting sets are 
` ellipses centered at (à, B). 


(45) 
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The simple regression model (41) can be generalized in many directions; 
the means £, may for example be polynomials in 1, of higher than the first 
degree (see Problem 30), or more complex functions such as trigonometric 
polynomials; or they may be functions of several variables, ¢;, u;, vj. Some 
further extensions will now be illustrated by a number of examples. 


Example 6. A variety of problems arise when there is more than one regression- 


line. Suppose that the variables X;, are independently normally distributed with 
common variance and means 


(46) iy = a; + Bt; (G= bhi i anb): 
The hypothesis that these regression lines have equal slopes 
H:B,= s =B, 


may occur for example when the equality of a number of growth rates is to be 
tested. The parameter space IIg has dimension s = 2b provided none of the sums 
E,(5,— t)? is zero; the number of constraints imposed by the hypothesis is 
r= b — 1. The minimum value of ZE(X,; — £j)? under Q is obtained by minimiz- 
ing E,(X,; — a; — Bt;;)? for each i, so that by (42), 


L(x- X,.)(4 - ti) 
b-n a à, 7 X. B. 


EG, - 4) 


Under H, one must minimize YE(X,; — &; — Bt, ^, which for any fixed £ leads 
to Mex Bt,. and reduces the sum of squares to ELIX =X) B(ty — tI. 


Minimizing this with respect to B, one finds 


; EEOy-3Xu-1) ^ &l.x-h. 


PARTIAL ET 


YX(t- t.) 
Since 


Xj- £ =X; -à,— [ = (X; E X.) - Bt; Y tj) 


and 


By Y E -(&- &) + t (B - Ê) ve (B, 5 b), - D 
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the rejection region (15) is 


X(À- B)'EG,- Y 0-0) 
EE x) Bil- t)| Ao - 2b) 2 


where the left-hand side under H has the F-distribution with b — land n» —-2b 
degrees of freedom. 
Since 


(47) 


LAL (4, - ti.) 


E(Àj) =B, and E) = “SST 


the noncentrality parameter of the distribution for an alternative set of B's is 
V = X((B, — BY (t, — t)! /o?, where B = E(À). In the particular case that the 
n, and the ¢,, are independent of i, B reduces to B = L8,/b. 

Example 7. The regression model (46) arises in the comparison of a number of 
treatments when the experimental units are treated as fixed and the unit effects uj; 
(defined in Chapter 5, Section 11) are proportional to known constants 1;;. Here t;j 
might for example be a measure of the fertility of the i, jth piece of land or the 
weight of the i, jth experimental animal prior to the experiment. It is then 


frequently possible to assume that the proportionality factor f, does not depend on 
the treatment, in which case (46) reduces to 


(48) £7 a, + Bt 
and the hypothesis of no treatment effect becomes 
H: q 7-42, 
The space II, coincides with II, of the previous example, so that s = b + 1 and 


Ê oh XXX; = X,J(t; A ti.) 


; à, = X,.— Bt,.. 
NETS j 
Minimization of D(X, — a — Bt,,)? gives 
* X,,— X.)(t,;— a 
'W YY(X,- x)(t - t.) ivy d. 


25» (19 — Ly à 


where X..— LLX,,/n, t..— EXt,,/n, n = En, The sum of squares in the numerator 
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of W* in (15) is thus 


S iu o 
YX(&-L5) = EE- x2 + BG t) Âl) 
The hypothesis. H is therefore rejected when 


EEN- X) + BG,- 5) Êl- |) Ab = 0 " 
XX[GG - X) - BG - 6) An - 5 - 1) 


(49) 


p 


where under H the left-hand side has the F-distribution with b — 1 andn-b-1 
degrees of freedom. 

The hypothesis H can be tested without first ascertaining the values of the ¢,;; it 
is then the hypothesis of no effect in a one-way classification considered in Section 
3, and the test is given by (19). Actually, since the unit effects u,, are assumed to be 
constants, which are now completely unknown, the treatments are assigned to the 
units either completely at random or at random within subgroups. The appropriate 
test is then a randomization test for which (19) is an approximation. 


Example 7 illustrates the important class of situations in which an 
analysis of variance (in the present case concerning a one-way classification) 
is combined with a regression problem (in the present case linear regression 
on the single “concomitant variable” 1). Both parts of the problem may of 
course be considerably more complex than was assumed here. Quite gener- 
ally, in such combined problems one can test (or estimate) the treatment - 
effects as was done above, and a similar analysis can be given for the 
regression coefficients. The breakdown of the variation into its various 
treatment and regression components is the so-called analysis of covariance. 


8. ROBUSTNESS AGAINST NONNORMALITY 


The F-test for the equality of a set of means was shown to be robust against 
nonnormal errors in Section 3. The proof given there extends without much 
change to the analysis of variance tests of Sections 5 and 6, but the situation 
is more complicated for regression tests. Reeth 

As an example, consider the simple linear-regression situation (41). More 
specifically, let U}, U,,... be a sequence of independent random variables 
with common distribution F, which has mean 0 and finite variance c?, and 


let 
Xm at Bt; + U- 


If F is normal, the distribution of B given by (42) is N(0, o 2/y(t; — D?) for 
all sample sizes and therefore also asymptotically. However, for nonnormal 
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F, the exact distribution of Ê will depend on the t's in a more complicated 
way. An asymptotic theory requires a sequence of constants /,,/5,.... A 
sufficient condition on this sequence for asymptotic normality of B can be 
obtained from the following lemma, which we shall not prove here but 
which is an easy consequence of the Lindeberg form of the central limit 
theorem. [See for example Arnold (1981, Theorem 10.3).] 


Lemma 3. Let Y,Y,... be independently identically distributed with 
mean zero and finite variance a?, and let Cj €5,... be a sequence of 
constants. Then a sufficient condition for Y? ,cY,/ \Lc? to tend in law to 
N(0, o?) is that 


max c? 
(50) EET as n oc. 
xg 
j=1 
The condition (50) prevents the c’s from increasing so fast that the last 
term essentially dominates the sum, in which case there is no reason to 
expect asymptotic normality. Applying the lemma to the estimator f of f, 
we see that 
Yt - « - Bu) - 1) 
B- B= i pt SRE FREE 
Ls, "d ) 


and it follows that 
(8 - B) X(t, - iy 
o 
tends in law to N(0, 1) provided 


(51) Cr AL ad ent. 
L(y i i) 


Example 8. The condition (51) holds in the case of equal spacing 1, = a + iA, 
but not when the t's grow exponentially, for example, when 1, = 2 (Problem 31). 


In case of doubt about normality we may, instead of relying on the above 
result, prefer to utilize tests based on the ranks of the X’s, which are exactly 
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distribution-free and which tend to be more efficient when F is heavy-tailed. 
Such tests are discussed in the nonparametric books cited in Section 3; see 
also Aiyar, Guillier, and Albers (1979). 

Lemma 3 holds not only for a single sequence c}, c5, ..., but also when 
the c’s are allowed to change with n so that they form a triangular array c;,, 
i=1,...,m, n=1,2,..., and the condition (51) generalizes analogously. 

Let us next extend (51) to arbitrary linear hypotheses with r = 1. The 
model will be taken to be in the parametric form (18) where the elements a; 
may depend on n, but s remains fixed. Throughout, the notation will 
suppress the dependence on n. Without loss of generality suppose that 
A'A = I, so that the columns of A are mutually orthogonal and of length 1. 
Consider the hypothesis 


H:0= Y bA-0 
j=l 
where the b’s are constants with Eb? = 1. Then 
6 = 6, = Eb, = Ed,X,. 
where by (18) 
(52) d,= Lab; 
By the orthogonality of 4, E d? = Eb; = 1, so that under H, 
E(8)-0 and Var (8) = o?. 
Thus, H is rejected when the t-statistic 
141 


saM 
Y(x - &/(n- 5 


(53) 


the denominator tends to c? in probability, 


I i i that 
t was shown in Section 3 tha to N(0, o?) provided 


and it follows from Lemma 3 that tends in law 
(54) maxd?— 0 as n- o. 


Under this condition, the level of the t-test is therefore robust against 
nonnormality. 
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So far, b = (b,,..., b) has been fixed. To determine when the level of 
(53) is robust for all b with Lb? = 1, it is only necessary to find the 
maximum value of d; as b varies. By the Schwarz inequality 


2 5 
4 - (Zab) Lo a 
5 


j= 


with equality holding when b, = a,,/ Exa. The desired maximum of d? 


5 2 
is therefore È ji js and 


s 

(55) max } a} >0 as noo 
i jal 

is a sufficient condition for the asymptotic normality of every ó,. 

The condition (55) depends on the choice of coordinate system in the 
parameter space, and in particular on the assumed orthogonality of A. To 
obtain a condition that is coordinate-free, consider an arbitrary change of 
coordinates B* = B~'8, where B is nonsingular. Then = AB = ABB* = 
A*B* with A* = AB. To be independent of the coordinate system, the 
condition on A must therefore be invariant under the group G of transfor- 
mations A > AB for all nonsingular B. It was seen in Example 3 of 
- Chapter 6 that the maximal invariant under G is P, = A(A‘A)~14’, so that 
the condition must depend only on P,. We are therefore looking for a 
function of P, which reduces to Y jj when the columns of A are 
orthogonal. In this case P, = AA’, and £ ja? is the ith diagonal element of 
P,. If II,, denotes the ijth element of P,, (55) is thus equivalent to the 
Huber condition 


(56) maxIl;— 0 as n— o, 
Li 


which is coordinate-free. 

If IL, < M, for all i = 1,..., n, then also II; < M, for all i and j. This 
follows from the fact (see Example 3 of Chapter 6) that there exists à 
nonsingular E with P — EE', on applying the Schwarz inequality to the 
ijth element of EE’. It follows that (56) is equivalent to 


(57) maxIl, 0 as n- oo. 
^J 


Theorem 3. Let X,=£,+U, (i 1,..., n), where the U’s are iid 
according to a distribution F with E(U,) = 0, Var(U;) = a?, and where for 
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each n the vector £ = (£y ..., £,) is known to lie in an s-dimensional linear 
subspace IL of R” given by (18) and satisfying (56). Then the size o (F) of 
the normal theory test given by (T) and (8) for testing H: € TL”, where 
II”) is any subspace of YI?" of fixed dimension s — r (0 <r x s), satisfies 
a,(F) > aasn- o. 


, Proof. lt was seen earlier that when (56) holds, the distribution of 
6, = Yb;B, tends to N(0, o?) for any b with £b; = 1. By the Cramér-Wold 
theorem [see for example Billingsley (1979), Theorem 29.4), this implies 
that f, ..., B, have a joint s-variate normal limit distribution with mean 0 
(under H) and covariance matrix o?I. Without loss of generality suppose 
that Ê; = 7,, where the n’s are given by the canonical form of Section 1. 
Then the columns of A are orthogonal and of length 1, and B; = Y, By 
standard multivariate asymptotic theory (Theorem 1.7 of TPE), the limit 
distribution of X1 Y? = Xj. ,B? under H is then that of a sum of squares 
of independent normal variables with means zero and variance o°, that is, 
o?x?, independent of F. The robustness of the level of (7) now follows from 
the fact, shown in Section 3, that the denominator of W* tends to o? in 
probability. 1 

For evaluating Il; it is helpful to note that £j, = Xj, IL, X, (i= 
1,..., n), so that I; is simply the coefficient of X, in £;, which must be 
calculated in any case to carry out the test. 

As an example, consider once more the regression example that opened 
the section. From (42), it is seen that the coefficient of X, in £, = â + Bt; is 
IL, = 1/n + (t; — i) /X(tj — D? and (56) is thus equivalent to the condi- 
tion (51) found earlier for this example. f 

As a second example, consider a two-way layout with m observations per 
cell, and the additive model £;;j, = E(Xjy) = wt a + B; (i =l, a; 
j=1,..., b), La, = E, = 0. Then $ijk = Xpt Xj Xs and it is seen 
that for fixed a and b, (56) holds as m  co- i 

The condition (56) guarantees asymptotic robustness for all linear hy- 
potheses II,, C IIg. If one is concerned only with a particular hypothesis, a 
weaker condition will suffice (Problem 40). 


9. SCHEFFÉ'S S-METHOD: A SPECIAL CASE 


If X,,..., X, are independent normal with common variance o? and 


expectations E(X;) = a + Bt, confidence sets for (a, B) were obtained in 


the preceding section. A related problem is that of determining confidence 
line ¿=a + ft, that is, functions 


bands for the whole regression 
L'(t; X), M'(t; X) such that 


(58) P(L( X) sat Br M X) for all t) = y. 
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The problem of obtaining simultaneous confidence intervals for a con- 
tinuum of parametric functions arises also in other contexts. In the present 
section, a general problem of this kind will be considered for linear models. 
Confidence bands for an unknown distribution function were treated in 
Section 13 of Chapter 6. 

Suppose first that X,,..., X, are independent normal with variance 
o2=1 and with means E(X,) = £j and that simultaneous confidence 
intervals are required for all linear functions Lu,£,. No enerality is lost by 
dividing Xu,£, and its lower and upper bound by feat , so that attention 
can be restricted to confidence sets 


(59) S(x): L(u; x) < Lug < M(u;x) forall ue U, 


where x, u denote both the vectors with coordinates x;, u; and the r X 1 
column matrices with these elements, and where U is the set of all u with 
Yu? = 1. The sets S(x) are to satisfy 


(60) P,[S(X)]=y forall € = ($, ---,£,)- 

Since u = (uj,...,u,) € U if and only if -u=(—m,..-, —u,) € U, 
the simultaneous inequalities (59) imply L(—u; x) € —Euj$, < M(—4% x) 
and hence 

-M(-u;x) < Xu s -L(-u;x) 
and 
max (L(u; x), - M(-u; x)) < Dug, < min(M(u; x), -L(-u; x)). 
Nothing is therefore lost by assuming that L and M satisfy 
(61) L(u; x) = -M(-u; x). 


The problem of determining suitable confidence bounds L(u; x) and 
M(u; x) is invariant under the group G, of orthogonal transformations 


G: gx = Qx, = QE (Q anorthogonal r x r matrix). 
Writing Lu,é, = u'£, we have 
g'S(x) = (Q£ L(u; x) < w < M(u; x) for all u € U} 
= (& L(u; x) < w(Q^£) < M(u; x) forall u € U) 


= (& L(Q^!u; x) < w < M(Q^!u; x) for all u € U}, 
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where the last equality uses the fact that U is invariant under orthogonal 
transformations of u. 
Since 


S(gx) = (£ L(u; Ox) < wt < M(u; Qx) forall u E€ U}, 
the confidence sets S(x) are equivariant under G, if and only if 
L(u;Qx)-L(Q^ux)  M(u, Qx) = M(Q^'u; x), 
or equivalently if 
(62) L(Qu; Ox) = L(u; x), M(Qu; Ox) = M(u; x) 
forall x,Q and u € U, 
that is, if L and M are invariant under common orthogonal transformations 


of u and x. Mee: 
A function L of u and x is invariant under these transformations if and 


only if it depends on u and x only through u’x, x'x, and u'u [Problem 
42(1)] and hence (since u'u = 1) if there exists A such that 


(63) L(u; x) = h(u'x, x'x). 


A second group of transformations leaving the problem invariant is the 
group of translations 


G,;:gx- xta, ge=Eta 


where x + a = (x + 4j... X, + 4,). An argument paralleling that lead- 
ing to (62) shows that L(u; x) is equivariant under G; if and only if 
[Problem 42(ii)] 


(64)  L(ux-*a)-L(ux)-* Zau, for all x, a, and u. 
The function A of (63) must therefore satisfy 
h[u'(x + a), (x + a) (x+ a))- h(u'x, x'x) + a'u 
forall a,x and u € U, 
and hence, putting x — 0, 
h(u'a, a'a) = a'u + h(0,0). 
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A necessary condition (which clearly is also sufficient) for S(x) to be 
equivariant under both G, and G, is therefore the existence of constants c 
and d such that 

S(x) = (£:Xuxi- es Lugs Yujx,+d forallue U}. 


From (61) it follows that c = d, so that the only equivariant families S(x) 
are given by 


(65) S(x) = G |Ew(x-£&)|se forall u € U). 

The constant c is determined by (60), which now reduces to 
(66) PA|XwX|sc for all u € U) zd 
By the Schwarz inequality (Zu, X < EX?, since Eu? = 1, and hence 
(67)  |XwX|sc forall we U ifand only if «Son ec: 
The constant c in (65) is therefore given by 
(68) P(xisc)-r 


In (65), it is of course possible to drop the restriction u € U by writing (65) 
in the equivalent form 


(69) S(x) = fë: (E u(x; - £)|s ef Lu? for all u}. 


So far attention has been restricted to the confidence bands (59). How- 
ever, confidence sets do not have to be intervals, and it may be of interest to 
consider more general simultaneous confidence sets 
(70) S(x): Y.u£, € A(u, x) for all u € U. 


For these sets, the equivariance conditions (62) and (64) become respectively 
(Problem 43) 


(71) A(Qu,Qx)=A(u,x) forall x,Q and u € U 


and 


(72) A(u,x+a)=A(u,x)+wa forall u,x,and a. 
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The first of these is equivalent to the condition that the set A(u, x) depends 
on u € U and x only through wx and x'x. On the other hand putting 
x = 0 in (72) gives 


A(u, a) = A(u,0) + u'a. 
It follows from (71) that A(u, 0) is a fixed set A; independent of u, so that 
(73) A(u, x) = A + u'x. 


The most general equivariant sets (under G, and G;) are therefore of the 
form 


(74) Yu(x-&)€A4 forall we U, 


where A = —4A,. 

We shall now suppose that r > 1 and then show that among all A which 
define confidence sets (74) with confidence coefficient > y, the sets (65) are 
smallest! in the very strong sense that if Ay = [— co, co] denotes the set (65) 
with confidence coefficient y, then Ay is a subset of A. 

To see this, note that if Y, = X, — $; the sets A are those satisfying 


(75) P(XwX€4A forall u € U) 2 v. 


Now the set of values taken on by Eu;y; for a fixed y = (X5. Y) aS U 
ranges over U is the interval (Problem 43) 


roy- [E x]. 


Let c* be the largest value of c for which the interval [— c, 
A. Then the probability (75) is equal to 


P(i(Y) c A) = P(Y) € Ee e*D- 


c] is contained in 


Since P(I(Y) € A) 2 7, it follows that c* > co, and this completes the 


proof. ; 

It is of interest to compare the simultaneous confidence intervals (65) for 
all Du,g,, u € U, with the joint confidence spheres for (£y... £) given by 
(41) of Chapter 6. These two sets of confidence statements are equivalent in 


the following sense. 

tA more general definition of smallness is due to Wijsman (1979). It has been pointed out to 
me by Professor Wijsman that his concept is equivalent to that of tautness defined by Wynn 
and Bloomfield (1971). 
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Theorem 4. The parameter vector (£y, . .., §,) satisfies E(X; — £j)" < e 
if and only if it satisfies (65). 


Proof. The result follows immediately from (67) with X, replaced by 
X, = $). 

Another comparison of interest is that of the simultaneous confidence 
intervals (69) for all u with the corresponding interval 


(76) s'(x) = (& Ew, - &)|s (Zu? } 


for a single given u. Since Xu,( X, — §;)/ Eu? has a standard normal 
distribution, the constant c’ is determined by P(x? < c^) = y instead of 
by (68). If r > 1, the constant c? = c? is clearly larger than c^ = cj. The 
lengthening of the confidence intervals by the factor c,/c, in going from 
(76) to (69) is the price one must pay for asserting confidence y for all Xu;£, 
instead of a single one. 

In (76), it is assumed that the vector u defines the linear combination of 
interest and is given before any observations are available. However, it often 
happens that an interesting linear combination Li,é, to be estimated is 
suggested by the data. The intervals 


(77) [Xà(x, - £)| s ey ya? 


with c given by (68) then provide confidence limits for Xà,£, at confidence 
level y, since they are included in the set of intervals (69). [The notation a; 
in (77) indicates that the u’s were suggested by the data rather than fixed in 
advance.] 


Example 9. Two groups. Suppose the data exhibit a natural split into a lower and 
upper group, say £,,..., £i, and £,,..., t; ,, with averages €_ and &,, and that 
confidence limits are required for Ẹ, — Ẹ_. Letting X_ = (X, +-+ * X,/k and 


d = (X; +--+ +X,_,)/(r - k) denote the associated averages of the X’s, we see 
that 


€—— ic 1 oe 1 1 
78 X,- X_- cy —+— - X,-X = 
(18) y "Vx rok Sh §.<X,-X_+c et I 
with c given by (68) provide the desired limits. Similarly 
(19) we Baap Smtr: é uif oie eel apart not 
ye on i Fae SETS Wr 
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provide simultaneous confidence intervals for the two group means separately, with c 
again given by (68). [ For a discussion of related examples and issues see Peritz (1965). 


Instead of estimating a data-based function Li,£;, one may be interested 
in testing it. At level a = 1 — y, the hypothesis Li,é; = 0 is rejected when 
the confidence intervals (77) do not cover the origin, i.e. when 


|Da.x;|> cf La? 


Equivariance with respect to the group G, of orthogonal transformations 
assumed at the beginning of this section is appropriate only when all linear 
combinations Lu,é, with u € U are of equal importance. Suppose instead 
that interest focuses on the individual means, so that simultaneous con- 
fidence intervals are required for £,, ..., £,. This problem remains invariant 
under the translation group G;. However, it is no longer invariant under G;, 
but only under the much smaller subgroup Go generated by the n! permuta- 
tions and the 2" changes of sign of the X’s. The only simultaneous intervals 
that are equivariant under Go and G, are given by [Problem 44(i)] 


(80) S(x) = (Ex, A && «x, + A for all i}, 
where A is determined by 
(81) P[S(X)] = P(max|Y| < 4) = Y 


with Y,,..., Y, being independent N(0, 1). 
These maximum-modulus intervals for the £'s can be extended to all 
linear combinations Xu,£, of the £'s by noting that the right side of (80) is 


equal to the set [Problem 45(ii)] 
(82) (& [Eu (x - [s AE u| foral u), 


which therefore also has probability y, but which is not equivariant under 


G,. A comparison of the intervals (82) with the Scheffé intervals (69) shows 
horter when Xuj£, = £, (ie. 


[Problem 44(iii)] that the intervals (82) are s: 
when u,=1 for j =i, and uj = 0 otherwise), but that they are longer for 
J 


example when u, = :** = 4r 


10. SCHEFFÉ'S S-METHOD FOR GENERAL 
LINEAR MODELS 


The results obtained in the preceding section for the simultaneous a 
tion of all linear functions Xu,£, when the common variance of the variab! * 
X, is known easily extend to the general linear model of Section 1. In the 
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canonical form (2), the observations are n independent normal random 
variables with common unknown variance o? and with means E(Y,) = 7, 
for i—1,...,r,r +1,...,s and E(Y) =0 for i= s * 1,..., n. Simulta- 
neous confidence intervals are required for all linear functions L7_,u;, with 


u € U, where U is the set of all u = (u,,..., u,) with Ej ,u? = 1. Invari- 
ance under the translation group Y; = Y, +a, i — r 1,...,s, leaves 
Y,,..., Y; Y,,,,..., Y, as maximal invariants, and sufficiency justifies re- 


stricting attention to Y = (Y,,..., ¥,) and S? = L”_, ,Y?. The confidence 
intervals corresponding to (59) are therefore of the form 


(8) L(u;y,S)< Dun, <M(u;y,S) forall we U, 
ii 


and in analogy to (61) may be assumed to satisfy 
(84) L(u; y, S) = -M(-u; y, S). 


By the argument leading to (63), it is seen in the present case that 
equivariance of L(u; y, S) under G, requires that 


L(u; y, S) = h(u'y, y'y, S), 


and equivariance under G, requires that L be of the form 
L(u; y, S) - 2 u,y, — e(S). 
i=l 


Since c? is unknown, the problem is now also invariant under the group of 
scale changes H 


G3: yj = by, (i= 1,...,r), S' 2 bS (b> 0). 


Equivariance of the confidence intervals under G, leads to the condition 
[Problem 45(i)] 


L(u; by, bS) = bL(u; y, S) forall b»0, 
and hence to 4 
bYuy,- (6S) = b[ Duy, e(s)]. 


or c(bS) = bc(S). Putting S = 1 shows that c(S) is proportional to S. 
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Thus 


L(uy,S)-YXuy-c$,  M(uiy,S)-Yu,,- dS, 


and by (84), c — d, so that the equivariant simultaneous intervals are given 
by 


(85) Yuwy-c$zXumsiuy-cS forall ueU. 
Since (85) is equivalent to 


2 
A. ",) en 


, 


the constant c is determined from the F-distribution by 


bebé essen WEA 
86)  B|—7—— s c = (Faas ct) =. 
(89), po S(n=s) r x r 


As in (69), the restriction u € U can be dropped; this only requires 


replacing c in (85) and (86) by c/Eu? = cy VarXu;Y, /a* ; j 
As in the case of known variance, instead of restricting attention to the 
confidence bands (85), one may wish to permit more general simultaneous 


confidence sets 
(87) Yun, € A(u; y, S). 


The most general equivariant confidence sets are then of the form [Problem 
45(ii)] 


(88) I €A forall ueEU, 


and for a given confidence coefficient, the set A is minimized by Ay = 
[—c, c], so that (88) reduces to (85). i . 

F P. i it is convenient to express the intervals (85) in ee 
the original variables X, and £;. Suppose as in Section 1 that X, "t "dbi H 
independently distributed as N(£;, o*), where £ = (5... spe n 
lie in a given s-dimensional linear subspace Ilgo (s < n). hod 
r-dimensional subspace of IIg (7 < $), let £, be the least squares estim 
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of the £'s under IIg, and let S? = E(X; — £,)?. Then the inequalities 


(89) 


var( Xo£;) y st bt Re var (Y) 


c? o 


Yo, - cS 


forall v € V, 


with c given by (86), provide simultaneous confidence intervals for Lv,é; for 
all v € V with confidence coefficient y. 

This result is an immediate consequence of (85) and (86) together with 
the following three facts, which will be proved below: 


G) Eum = L198; then E; .,uY, = XD 

G) EY = EpaX = È 
To state (iii), note that the ņ’s are obtained as linear functions of the £'s 
through the relationship 
(90) (m «06 2:630, -,0) = C(b.. Sn)" 


where C is defined by (1) and the prime indicates a transpose. This is seen 
by taking the expectation of both sides of (1) For each vector u = 
(15, ... , u,), (90) expresses Du,n, as a linear function Lol |, of the £'s. 


(iii) As u ranges over r-space, v™ = (v(,... , v?) ranges over V. 


Proof of (i). Recall from Section 2 that 


Sayers E Y 


j=l i=1 j=s+1 


Since the right side is minimized by 7, = Y, and the left side by £; = £j, this 
shows that 


(X, -- Y,0---0/ 2 c(& --- &)’, 


and the result now follows from comparison with (90). 


Proof of (ii). This is just equation (13). 
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Proof of (iii). Since n; = 7,06, we have Lum; = Xvf?£, with vj? 
= YrLQu;c;. Thus the vectors v? — (v(?,..., v(?) are linear combina- 
tions, with weights u,,..., u,, of the first r row vectors of C. Since the space 
spanned by these row vectors is V, the result follows. 

The set of linear functions Xu,£;, v € V, for which the interval (89) does 
not cover the origin—that is, for which v satisfies 


^ Var | 2,oj6; 
(91) [Evé |> cs Ea 
—is declared significantly different from 0 by the intervals (89). Thus (91) is 
a rejection region at level a = 1 — y of the hypothesis H : Xv£, = 0 for all 
v € V in the sense that H is rejected if and only if at least one v € V 
satisfies (91). If IL, denotes the (s — r)-dimensional space of vectors 
v € Ilo which are orthogonal to V, then H states that £ € II,, and the 
rejection region (91) is in fact equivalent to the F-test of H:£€ II, of 
Section 1. In canonical form, this was seen in the sentence following (85). 

To implement the intervals (89) in specific situations in which the 
corresponding intervals for a single given function Lu,é, are known, it is 
only necessary to designate the space V and to obtain its dimension r, the 
constant c then being determined by (86). 

Example 10. All contrasts. Let Xj (j = 1,- ni j=1,...,5) be indepen- 
dently distributed as N(£,, o^), and suppose V is the space of all vectors v = 
(v,,...,0,) satisfying 


(92) Yo, - 0. 


Any function Ev;¢; with v € V is called a contrast among the £j. The set of 

contrasts includes in particular the differences £, — $- discussed in Example 9. The 

space TIo is the set of all vectors (£,.--» & £o 66) and has dme. 
sion s, while V is the subspace of vectors IIp that are orthogonal to (1,...,1) an 

hence has dimension r = s — 1. It was seen in Section 3 that &, = X, and if the 

vectors of V are denoted by 

( Wy Ww; We W Ws » | 

— enw 25 grey , 

s 


ra 
n nm. N2 ucl 


the simultaneous confidence intervals (89) become (Problem 47) 


nn 


<E wg sXwX- cS p 


(93) YwX.- cS 


forall (v) satisfying Yw-0, 


with 5? = YY(X,, — X". 
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In the present case the space II, is the set of vectors with all coordinates equal, 
so that the associated hypothesis is H: £j = --- = £,. The rejection region (91) is 
thus equivalent to that given by (19). 

Instead of testing the overall homogeneity hypothesis H, we may be interested in 
testing one or more subhypotheses suggested by the data. In the situation corre- 
sponding to that of Example 9 (but with replications), for instance, interest may 
focus on the hypotheses H; :£, = ++ = £j, and Ee wenn 6, A level a 
simultaneous test of H, and H, is given by the rejection region 


EZos(x-XP)Ak-).. — EOm(X— XO) /(s - k - 1) 
SEEN) ; S1/(n —5) 3 


where E), E, X, X® indicate that the summation or averaging extends over the 
sets (ij,... ip) and (jj,..., j-p) respectively, S? = LE(X,, — X),a-1-7» 
and the constant C is given by (86) with r — s and is therefore the same as in (19), 
rather than being determined by the F, ,, , and F, , ,,,.., distributions. The 
reason for this larger critical value is, of course, the fact the H, and H, were 
suggested by the data. The present procedure is an example of Gabriel's simulta- 
neous test procedure mentioned in Section 4. 


C; 


Example 11. Two-way layout. As a second example, consider first the additive 
model in the two-way classification of Section 5 or 6, and then the more general 
interaction model of Section 6. 

. Suppose X,, are independent N(£;j, 02) (i= 1,...,a; f= 1,--., b), with $y 
given by (32), and let V be the space of all linear functions Xw;a; = Ew, (5,.— 5.) 
As was seen in Section 5, s = a + b — 1. To determine r, note that V can also be 
represented as Y.,.. w;£,. with Dw, = 0 [Problem 46(i)], which shows that r = a ~ 1. 
The least-squares estimators £; were found in Section 5 to be j= X+ X X, 
so that £,.— X, and S? = YE(X; — X.— Xj + X..)?. The simultaneous confidence 
intervals (89) therefore can be written as 


2 S d 
XwX- ey < Dwg. <DwX+ Sy = 


for all w with }, w; = 0. 


i=l 


If there are m observations in each cell, and the model is additive as before, the only 
changes required are to replace X. by X,., S? by ELE(X,j, — X; Xj X) * 
and the expression under the square root by Lw?/bm. b 

Let us now drop the assumption of additivity and consider the general linear 
model £j, = p + a; + B; + yj, with p and the a's, B's, and y's defined as in 
Section 6. The dimension s of Ig is then ab, and the least-squares estimators of 
the parameters were seen in Section 6 to be 


B-Xlso à-X—XoB-2X.X. 


Yy = X7 X Xj X. 
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The simultaneous intervals for all Xw,a;, or for all Xw£.. with Zw, = 0, are 
therefore unchanged except for the replacement of s =A a X. Xj 
X...) by S? 2 (X, — Xj) andotn-sen-a-b*lbyn-scn-ab 
— (m — l)ab in (86). 

Analogously, one can obtain simultaneous confidence intervals for the totality of 
linear functions Dw, y, ;, or equivalently the set of functions Yw,,;,. for the totality 
of w's satisfying D,w,, = L,w,; = 0 [Problem 46(ii), (iii)]. 


Example 12. Regression line. As a last example consider the problem of obtain- 
ing confidence bands for a regression line, mentioned at the beginning of the 
section. The problem was treated for a single value fọ in Chapter 5, Section 8 (with 
a different notation) and in Section 7 of the present chapter. The simultaneous 
confidence intervals in the present case become 


(94) â&+ Ê S PX Ga hee + Bt 
ag |e ee V 
a+ Bt —oc n (y 


Teva epu py ve 
n yu - N 


cathoa|ts 


where & and are given by (33), 
s -E(x-a- hj DK - XY - FEQ - 0 


and c is determined by (86) with r=s= 2. This is the Working-Hotelling 
confidence band for a regression line. 


At the beginning of the section, the Scheffé intervals were derived as the 
only confidence bands that are equivariant under the indicated groups. If 
the requirement of equivariance (particular under orthogonal transforma- 


tions) is dropped, other bounds exist which are narrower for certain sets of 


vect f being wider for others [Problems 45(iii) and 68]. A 
ceti RR ENES bset is described by 


general method that gives special emphasis to a given sul desc 
Richmond (1982). Some optimality results not requiring equivariance in 
instead permitting bands which are narrower for some values of t d e 
expense of being wider for others are provided, among others, by Bol e 
(1973), Cima and Hochberg (1976), Richmond (1982), Naiman (1984a, b), 
and Piegorsch (19852, b). If bounds are required only for a subset, it may be 
possible that intervals exist at the prescribed confidence level, which are 

fé intervals. This is the case for example 


uniformly narrower than the Schel ; o 
for the intervals (94) when t is restricted to a given finite interval. For a 


discussion of this and related problems, and references to the literature, see 
for example Wynn and Bloomfield (1971) and Wynn (1984). 
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11. RANDOM-EFFECTS MODEL: ONE-WAY 
CLASSIFICATION 


In the factorial experiments discussed in Sections 3, 5, and 6, the factor 
levels were considered fixed, and the associated effects (the p’s in Section 3, 
the a's, B's and y's in Sections 5 and 6) to be unknown constants. 
However, in many applications, these levels and their effects instead are 
(unobservable) random variables. If all the effects are constant or all 
random, one speaks of fixed-effects model (model I) or random-effects model 
(model IT) respectively, and the term mixed model refers to situations in 
which both types occur. Of course, only the model I case constitutes a linear 
hypothesis according to the definition given at the beginning of the chapter. 
In the present section we shall treat as model II the case of a single factor 
(one-way classification), which was analyzed under the model I assumption 
in Section 3. 

As an illustration of this problem, consider a material such as steel, 
which is manufactured or processed in batches. Suppose that a sample of 
size n is taken from each of s batches and that the resulting measurements 
Xj (j =1,...,n; i= 1,..., s) are independently normally distributed with 
variance o? and mean £, If the factor corresponding to i were constant, 
with the same effect a, in each replication of the experiment, we would have 


&7H- a, (Xa, = 0) 
and 


Xj = b+ a;,+ Uj 


where the U;, are independently distributed as N(0, 02). The hypothesis of 
no effect is 5, = -+- = £, or equivalently a, = «+: =a, = 0. However, 
the effect is associated with the batches, of which a new set will be involved 
in each replication of the experiment; and the effect therefore does not 
remain constant. Instead, we shall suppose that the batch effects constitute à 
sample from a normal distribution, and to indicate their random nature we 
shall write A; for a, so that 


(95) Xj = b+ A+ Uj 


The assumption of additivity (lack of interaction) of batch and unit effect, 
in the present model, implies that the A’s and U’s are independent. If the 
expectation of A, is absorbed into p, it follows that the A's and U's are 
independently normally distributed with zero means and variances c; and 
c? respectively. The X’s of course are no longer independent. 
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The hypothesis of no batch effect, that the A’s are zero and hence 
constant, takes the form 


H: o} =0. 


This is not realistic in the present situation, but is the limiting case of the 
hypothesis 


9; 
H(A): EH < åo 


that the batch effect is small relative to the variation of the material within a 
batch. These two hypotheses correspond respectively to the model I hy- 
potheses Xa? = 0 and La?/o? < Ay. 

To obtain a test of H(Aọ) it is convenient to begin with the same 
transformation of variables that reduced the corresponding model I problem 
to canonical form. Each set (X,,.-., Xin) is subjected to an orthogonal 
transformation ¥,, = Y7..,c, X, such that Y, = Vn X,.. Since e, = 1/ vn 
for k = 1,...,n (see Example 3), it follows from the assumption of ortho- 
gonality that 17 .,c;, = 0 for j = 2,..., n and hence that Y,; = Ek=1CjkUik 
for j > 1. The Y,, with j > 1 are therefore independently normally distrib- 
uted with zero mean and variance o°. They are also independent of U,, since 
(Vn U,. Y; ... Y} = C(U, Ua... Uin) (a prime indicates the transpose ofa 
matrix). On the other hand, the variables Y; = Yn X,= Vn (+ Aj + U,.) 
are also independently normally distributed but with mean yn. p. and vari- 
ance o? + no}. If an additional orthogonal transformation is made from 
(Y... Y4) to (Zines -» Za) such that Zi = Vs Y. the Z's are indepen- 
dently normally distributed with common variance o? + n9; and means 
E(Z,,) = Vsnp and E(Z,) = 0 for i > 1. Putting Z= Y; for j > 1 for 
the sake of conformity, the joint density of the Z's is then 


96) (25) "2o 0-D:(0? + no?) "^ 
7 


1 : T 
xen rra le - Venu) + Zal -33L L| 
i=2 


2(0? + no) 


Th ting H(A,) is invariant under addition of an arbitrary 
d eee he remaining Z’s as a maximal set of 


Constant to Z,,, which leaves t à Aas 
invariants, These constitute samples of size s(n = baa s ie A Hs 
normal distributions with means zero and variances o and 71? = o^ + noj. 
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The hypothesis H(A ) is equivalent to 77/9? < 1+ Aon, and the problem 
reduces to that of comparing two normal variances, which was considered in 
Example 6 of Chapter 6 without the restriction to zero means. The UMP 
invariant test, under multiplication of all Z,, by a common positive con- 
stant, has the rejection region 


1 S;/(s - 1) 


(37) img Aon. S?/(n - 1)s 


mc. 


where 


S}= LZ} md $?-YlLZj- Y LY. 


i-2 i-1j-2 i=1 j=2 


The constant C is determined by 


[Pov dy = a. 


Since 


n n 
EYj-YP- hUj-aUl 
je j=l 


and 


s "s 
2 Zi- Zzi= EYi-sYh 


i=l i=l 


the numerator and denominator sums of squares of W*, expressed in terms 
of the X's, become 


e p» X.) and S?= y mie ee ae 


i-1 i=1 j=1 


In the particular case A, = 0, the test (97) is equivalent to the corre- 
sponding model I test (19), but they are of course solutions of different 
problems, and also have different power functions. Instead of being distrib- 
uted according to a noncentral x?-distribution as in model I, the numerator 
sum of squares of W* is proportional to a central x?-variable even when the 
hypothesis is false, and the power of the test (97) against an alternative 


7.11] RANDOM-EFFECTS MODEL: ONE-WAY CLASSIFICATION 421 
value of A is obtained from the F-distribution through 


B(A) = P{W* > C}= fison Fio 0 dr 
EP TI 


The family of tests (97) for varying A, is equivalent to the confidence 
statements 


(98) A7 | Cs*/@=1)s > 


1| SI sm 

| 4/(s=1) | BON 
The corresponding upper confidence bounds for A are obtained from the 
tests of the hypotheses A > Ag. These have the acceptance regions W* > C’, 
where W* is given by (97) and C’ is determined by 


0 
f. F5 genre esas 


and the resulting confidence bounds are 


1[ s2/(6 - 1) a 
(99) As ER 


Both the confidence sets (98) and (99) are equivariant with respect to the 
group of transformations generated by those considered for the testing 
problems, and hence are uniformly most accurate equivariant. A 
When A is negative, the confidence set (A, œ) contains all possible 
values of the parameter A. For small A, this will happen with high 
probability (1-« for A = 0), as must be the case, since A is then required 
to be a safe lower bound for a quantity which is equal to or near zero. Even 
more awkward is the possibility that A is negative, so that the confidence set 
(— co, A) is empty.* An interpretation is suggested by the fact that this 
occurs if and only if the hypothesis A > Ay is rejected for all positive values 
of Ay. This may be taken as an indication that the assumed model is not 
appropriate,* although it must be realized that for small A the probability of 


the event A < 0 is near a even when the assumptions are satisfied, so that 


this outcome will occasionally be observed. Nea 
The tests of A < Ay and A = A, are not only UMP invariant but also 
UMP unbiased, and UMP unbiased tests also exist for testing 4 = Ay 


further at the end of Chapter 10, Section 4. 


M ts are discussed z 
Such awkward confidence se! ternative models, see Smith and Murray 


* For a discussion of possibly more appropriate al 
(1984), 
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against the two-sided alternatives A # As. This follows from the fact that 
the joint density of the Z’s constitutes an exponential family. The con- 
fidence sets associated with these three families of tests are then uniformly 
most accurate unbiased (Problem 48). That optimum unbiased procedures 
exist in the model II case but not in the corresponding model I problem is 
explained by the different structure of the two hypotheses. The model II 
hypothesis o = 0 imposes one constraint, since it concerns the single 
parameter o. On the other hand, the corresponding model I hypothesis 
Ys_,a? = 0 specifies the values of the s parameters a,,...,@,, and since 
s — 1 of these are independent, imposes s — 1 constraints. 

A UMP invariant test of A < A does not exist if the sample sizes n, are 
unequal. An invariant test with a weaker optimum property for this case is 
obtained by Spjetvoll (1967). 

Since A is a ratio of variances, it is not surprising that the test statistic 
W* shares the great sensitivity to the assumption of normality found in 
Chapter 5, Section 4 for the corresponding two-sample problem. More 
robust alternatives are discussed, for example, by Arvesen and Layard 
(1975). 


12. NESTED CLASSIFICATIONS 


The theory of the preceding section does not carry over even to so simple a 
situation as the general one-way classification with unequal numbers in the 
different classes (Problem 51). However, the unbiasedness approach does 
extend to the important case of a nested (hierarchical) classification with 
equal numbers in each class. This extension is sufficiently well indicated by 
carrying it through for the case of two factors; it follows for the general case 
by induction with respect to the number of factors. 

Returning to the illustration of a batch process, suppose that a single 
batch of raw material suffices for several batches of the finished product. Let 
the experimental material consist of ab batches, b coming from each of a 
batches of raw material, and let a sample of size n be taken from each. Then 
(95) becomes 


(100) Xj = b +A; + By + Ujk 


where A, denotes the effect of the i th batch of raw material, B;; that of the 
jth batch of finished product obtained from this material, and U;,, the effect 
of the kth unit taken from this batch. All these variables are assumed to be 
independently normally distributed with zero means and with variances 04> 
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o3, and c? respectively. The main part of the induction argument consists in 
proving the existence of an orthogonal transformation to variables Z,;, the 
joint density of which, except for a constant, is 


1 a 
(101) exp} — ) (nns vabnu)’ + Bah) 


2(07 + nog + bno; S 


Indc ro 
= za- = zek 
2(0? + nog) i=1 j=2 Jt d 20? i=l fal k=2 

As a first step, there exists for each fixed j, j an orthogonal transforma- 
tion from (X, - +» Xijn) to (Ys -+> Y, jn) such that 


Yia = Yn X= np + Yn (A, + By t U,,.). 


As in the case of a single classification, the variables Y; with k » 1 
depend only on the U’s, are independently normally distributed with zero 
mean and variance o?, and are independent of the Uj.. On the other hand, 
the variables Y,, have exactly the structure of the Y; in the one-way 
classification, 


Yi =p +A; +t ij 


where p’ = Vnu, A; = Vn A, Uj = Yn (Bj, + Uy), and where the variances 
of A’ and Uj, are of? = no, and o"? = a? + nj, respectively. These vari- 
ables can therefore be transformed to variables Z; 1 whose density is gms 
by (96) with Z;; in place of Z;;.- Putting Z,;, = Yin for k > 1, the join 
density of all Z,,, is then given by (101). : 

Two hypoticsei of interest can be tested on the basis of (101)— 
H,: o2/(o? + no2) < Ao and H;: 02/0? < Ao, which state that one or the 
other of the classifications has little effect on the outcome. Let 


a as iD m "p Un 5 

i= L ZA - Z, S= LE L Zip 
aS Ez 2 p L bi j=1 j=l k=2 
one is tempted to eliminate S? through invariance 
under multiplication of Z;;, for k > 1 by an arbitrary ere cd 
these transformations do not leave (101) invariant, since ey diee 
preserve the fact that o? is the smallest of the three variances , St o 
and o? + no2+ bna?. We shall instead consider the problem from 

B i 


To obtain a test of Hj, 
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point of view of unbiasedness. For any unbiased test of H;, the probability 
of rejection is « whenever o2/(o? + nag) = Ao, and hence in particular 
when the three variances are o2, 72, and (1 + bno) rà for any fixed 7 and 
all o? < rê. It follows by the techniques of Chapter 4 that the conditional 
probability of rejection given S? = s? must be equal to a for almost all 
values of s2. With S? fixed, the joint distribution of the remaining variables 
is of the same type as (101) after the elimination of Z,,,, and a UMP 
unbiased conditional test given S? = s? has the rejection region 


1 Si[(a - 1) 


102 Worm ee OS 256i. 
02) 1 1+ dnd, Sab — l)a : 


Since S2 and S2 are independent of $°, the constant C, is determined by 
the fact that when 02/(a? + nog) = A, the statistic W,* is distributed as 
F,., (5-14 and hence in particular does not depend on s. The test (102) is 
clearly unbiased and hence UMP unbiased. 

An alternative proof of this optimality property can be obtained using 
Theorem 7 of Chapter 6. The existence of a UMP unbiased test follows 
from the exponential family structure of the density (101), and the test is the 
same whether 7? is equal to o? + nož and hence > o°, or whether it is 
unrestricted. However, in the latter case, the test (102) is UMP invariant and 
therefore is UMP unbiased even when 7? > 0°, 

The argument with respect to H, is completely analogous and shows the 
UMP unbiased test to have the rejection region 


1 Sàl(b — 1)a 


: z C, 
I*aÀ, S(n-1)ab ^ 


where C, is determined by the fact that for 02/0? = Ag, the statistic W;* is 
distributed as Fr , (as 
It remains to express the statistics S2, S2, and S? in terms of the X’s. 


M the corresponding expressions in the one-way classification, it follows 
that 


(103) w= 


a 
si = Y Zi - Za = OY (Yin - Ys 
M 


a b 
si- X|Xz- 2| EEO n2. 


i=1|j=1 
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and 


a b n n 
s-yY|5vxà-Xà >| uh mu | 
k=1 


i=1 j-1lk-1 io cs 


Hence 
(104) S7 = n E (a Xu) Si-1EEQG- %-) 


St XXXGu T XN 


It is seen from the expression of the statistics in terms of the Z’s that 
their expectations are E[S2/(a — 1)] = 0? + nog + bno}, E[S3/(b — 1)a] 
= c? + nj, and E[S?/(n — 1)ab] = o°. The decomposition 


EEEX p- X) = Si+ Sh +S? 
therefore forms a basis for the analysis of the variance of X; jx» 


Var( X;jx) =o} + of + o? 


by providing estimates of the components of variance a}, of, and o”, and 
tests of certain ratios of these components. 

Nested two-way classifications also occur as mixed models. Suppose for 
example that a firm produces the material of the previous illustrations in 


ffect of the ith plant (which is fixed, since 


different plants. If a; denotes the e! i 
the plants do not change in the replication of the experiment), B,, the batch 


effect, and U,;, the unit effect, the observations have the structure 


(105) Xi -ptat Bjt U- 
i i i f the 
Instead of reducing the X’s to the fully canonical form in terms of th 
Z's as before, it is convenient to carry out only the reduction to the oe 
(such that Y, = yn Xj) and the first of the two transformations which take 
the Y's into: the Z’s. If the resulting variables are denoted by Wiji they 
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satisfy Wa, = VB Y,» Wijk = Y; for k > 1 and 


a a b a b n 
D (Wai — Wai) T Si, ». 2) Wir ti Sis > » z Wii, =s? 
i=l i=1 j=2 i=l j=l k=2 


where S2, S2, and S? are given by (104). The joint density of the W’s is, 
except for a constant, 


(106) l- Il Y (wai E- aj) nd 3 » | 


2 2 
o? + nog) Vii ie1je-2 


LEE Ew 
v sari wi . 
207 i=l j=l k=2 te 


This shows clearly the different nature of the problem of testing that the 
plant effect is small, 


2 
H:a,= ++: =a,=0 or go. ES 
c^ + nog 
and testing the corresponding hypothesis for the batch effect: 02/0? < Ao. 
The first of these is essentially a model I problem (linear hypothesis). As 
before, unbiasedness implies that the conditional rejection probability given 
S? = s? is equal to a ae. With S? fixed, the problem of testing H isa 
linear hypothesis, and the rejection region of the UMP invariant conditional 
test given S? = s? has the rejection region (102) with A, = 0. The constant 
C, is again independent of S?, and the test is UMP among all tests that are 
both unbiased and invariant. A test with the same property also exists for 
testing H'. Its rejection region is 


S3(a- 1) 


EC 
S2l(b — 1)a 


where C' is determined from the noncentral F-distribution instead of, a5 
before, the (central) F-distribution. 

_ On the other hand, the hypothesis 02/0? < Ay is essentially model II. It 
is invariant under addition of an arbitrary constant to each of the variables 
W,,, which leaves Ya Wa and Sasha Wide as maximal in- 
variants, and hence reduces the structure to pure model II with one 
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classification. The test is then given by (103) as before. It is both UMP 
invariant and UMP unbiased. 

A two-factor mixed model in which there is interaction between the two 
factors will be considered in Example 2 of Chapter 8. Very general mixed 
models (containing general type II models as special cases) are discussed, 
for example, by Harville (1978), J. Miller (1977), and Brown (1984), but see 
the note following Problem 63. 

The different one- and two-factor models are discussed from a Bayesian 
point of view, for example, in Box and Tiao (1973) and Broemeling (1985). 
In distinction to the approach presented here, the Bayesian treatment also 
includes inferences concerning the values of the individual random compo- 
nents such as the batch means £, of Section 11. 


13. PROBLEMS 


1. Expected sums of squares. The expected values of the numerator and de- 
nominator of the statistic W* defined by (7) are 


2. Noncentral x?-distribution*. 


(i) If X is distributed as N(V, D), the probability density, SS Xe is 
pi (o) = Eg-oP, fas (0), where Piy) = (7/2) e une’ /k! and 
where fj,,; is the probability density of a x^-variable with 2k + 1 


degrees of freedom. Md 
(ii) Let Y;,...,Y, be independently normally distributed with unit variance 
and means q,...,7,. Then U - Y? is distributed according to the 
noncentral x?-distribution with r degrees of freedom and noncentrality 


parameter 4? = Xj. y], which has probability density 
oo 
(107) py) "m y PU) fra"): 
k=0 


ave the same meaning as in (i), so that the 


h 
Here P, (V) and fear CU) AY o tributions with Poisson weights 


distribution is a mixture of x 


[G): This is seen from 
ei to e c) 


A ed E 
[^4 v) 2y2m0 


s reviewed in Chapter 28 of Johnson 


2. including tables, i 
Ad d Owen (1984), and in Tiku (19852). 


“The literature on noncentral x ^ 
r, Rosenstein, ant 


and Kotz (1970, Vol. 2), in Chou, Arthu 
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by expanding the expression in parentheses into a power series, and using the 
fact that T(2k) = 274 !T(K)T(k + lys. 
(ii): Consider an orthogonal transformation to Z,,..., Z, such that Z, = 
En; Y,/y. Then the Z's are independent normal with unit variance and means 
E(Z,) = and E(Z,) = 0 for i > 1] 

3. Noncentral F- and beta-distribution* Let Y,,..., Y,; Y, ,1,..., Y, be indepen- 


dently normally distributed with common variance o^ and means E(Y;) = 
ni = l; 7s EŒ) 70 5 1... n). 


(i) The probability density of W = ELaY2/EL, Y) is given by (6). The 
distribution of the constant multiple (n — s)W/r of W is the noncentral 
F-distribution. 

Gi) The distribution of the statistic B = D/_,¥?/(/-1¥? + Ej-.. LY?) is the 
noncentral beta-distribution, which has probability density 


(108) E Osee 
where 
Q9) —gj0)- X wa PL) si 


is the probability density of the (central) beta-distribution. 


4. (i) The noncentral x? and F distributions have strictly monotone likelihood 
ratio. 

Gi) Under the assumptions of Section 1, the hypothesis H’: J^ < y$ (o > 9 
given) remains invariant under the transformations G, (i — 1,2,3) that 
were used to reduce H : y = 0, and there exists a UMP invariant test with 
rejection region W > C’. The constant C’ is determined by Pj, {W > C à) 
— a, with the density of W given by (6). 


[(): Let f(z) = Eg. ob, z^ /E-9a,z^ where the constants ax, b, are > 0 and 


Ya, z^ and Y,b, z* converge for all z > 0, and suppose that b, /a, < by +1/4k+1 
for all k. Then 


YX(n- k)(a,b, — a,b.) z^"! 


f(z)- E " 


k=0 


is positive, since (n — k)(a,b, — a,b,) > 0 for k < n, and hence f is increas- 
ing] 


*For literature on noncentral F, see Johnson and Kotz (1970, Vol. 2) and Tiku (1985b). 
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Note. The noncentral x°- and F-distributions are in fact STP, [see for 
example Marshall and Olkin (1979) and Brown, Johnstone and MacGibbon 
(1981)], and there thus exists a test of H : V = yọ against vj # Yọ which is 
UMP among all tests that are both invariant and unbiased. 


5. Best average power. 


(i) Consider the general linear hypothesis H in the canonical form given by 
(2) and (3) of Section 1, and for any 7,,,,...,7,, 9, and p let S = 
S(n,,1,.-. Mss 9; p) denote the sphere {(m,,..., n,): E -im/e? = p^). 
If B, (m, ..., ns, 0) denotes the power of a test $ of H, then the test (9) 


maximizes the average power 
[Bins ns) dA 
ZS bre womi nr 
faa 
s 


for every 7,415---» Ms, 9, and p among all unbiased (or similar) tests. 

Here dA denotes the differential of area on the surface of the sphere. 
(ii) The result (i) provides an alternative proof of the fact that the test 1) is 

UMP among all tests whose power function depends only on L211) /97. 


((i): if U = Et, Y2, V= EL, Y?, unbiasedness (or similarity) implies that 
the conditional probability of rejection given Y, ,;,..., Y, and U + V equals a 
a.e. Hence for any given ,,1,-.., Ms» 9, and p, the average power is maxi- 
mized by rejecting when the ratio of the average density to the density under H 
is larger than a suitable constant C(y,41,---» Js, u + v), and hence when 


5 0 + 
"— = fer È 2) dA > Cras ut v). 


i=l 


As will be indicated below, the function g depends on )j,.-- dr only through 
u and is an increasing function of u. Since under the hypothesis U/(U a eg 
independent of Y, ,,..., Y, and U + V, it follows that the test Heus y O 
The exponent in the integral pun le, ee n^ I a 

ucos B)/a, where B is the angle (0 < m) betwe 1.5, My) anc 
bus T. Mns d the symmetry of the sphere, this is apenas a ds 
replaced by the angle y between (m;--., 0,) and anainn e 2 
This shows that g depends on the y’s only through u; for fixe n 
denote it by h(u). Let S’ be the subset of S in which 0 < y < 7/2. 


h(u) = [fe peost) to anteni 


which proves the desired result.] 
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Use Theorem 8 of Chapter 6 to show that the F-test (7) is a-admissible against 
Q’: y >y for any y; > 0. 


Given any y, > 0, apply Theorem 9 and Lemma 3 of Chapter 6 to obtain the 
F-test (7) as a Bayes test against a set Q' of alternatives contained in the set 
0«y zy. 

Section 2 
Under the assumptions of Section 1 suppose that the means £, are given by 


s 


bz Ya; jo 


yb 


where the constants a;; are known and the matrix A = (4,;) has full rank, and 
where the f, are unknown parameters. Let 0 = Ej-ej; be a given linear 
combination of the £. 


) If Ê, denotes the values of the 8, minimizing E(X, — £) and if b= 
De, = Y/.,d,X, the rejection region of the hypothesis He 
0 = 6, is 


[t [E «x4 
(XQ - 8) An-s) 


where the left-hand side under H has the distribution of the absolute 
value of Student's ¢ with n — s degrees of freedom. 
(ii) The associated confidence intervals for 0 are 


TNR ar tect ote 


with k = C/d?. These intervals are uniformly most accurate equi- 
variant under a suitable group of transformations. 


(110) 


>Q 


[(i): Consider first the hypothesis @ = 0, and suppose without loss of general- 
ity that @ = B;; the general case can be reduced to this by making a lin- 
ear transformation in the space of the B's. If a;,..-, 2; denote the column 
vectors of the matrix A which by assumption span llo then 
E Bia o +B,a,, and since E is in Iq, also Ê = ĝia, + -+° +B,a,. The 
space II,, defined by the hypothesis B; = 0 is spanned by the vectors 2,---» as 
and also by the row vectors ¢),...,¢, of the matrix C of (1), while c, 1$ 
orthogonal to IL,. By (1), the vector X is given by X = Ef-1Y;ci» and its 


projection £ on Tg therefore satisfies é = E; _ Y, c. Equating the two expres- 
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sions for é and taking the inner product of both sides of this equation with c, 
gives Y, = BM yancy;, since the c's are an orthogonal set of unit vectors. 
This shows that Y, is proportional to B, and, since the variance of Y, is the 
same as that of the X's, that |¥,| = IB ea? . The result for testing B = 0 
now follows from (12) and (13). The test for 8, = f? is obtained by making 
the transformation X* = X, — aj f. 

(ii): The invarianee properties of the intervals (111) can again be discussed 
without loss of generality by letting 0 be the parameter £,. In the canonical 
form of Section 1, one then has E(Y,) = m = Mi with [A| = 1/ (La? while 
725. N, do not involve £,. The hypothesis B; = B is therefore equivalent to 
mı =n? with n? — A8?. This is invariant (a) under addition of arbitrary 
constants to Y,..., Y,; (b) under the transformations Y* = —(Y, — n$) + n}; 
(c) under the scale changes Y,* = cY, (i = 2,...,n), Yt — $* = c(Y, — m1). 
The confidence intervals for 0 = £, are then uniformly most accurate equi- 
variant under the group obtained from (a), (b), and (c) by varying v] 

Let X,; (j= l...,m;) and Y (k= 1,...,n,) be independently normally 
distributed with common variance o? and means E(X,,) = $; and E(Y;;) = $; 
+ A. Then the UMP invariant test of H : A = 0 is given by (110) with 0 = A, 
0, = 0 and 


rR Xi) Y x,* Lu -0) 

i ja - 

ES = 
N, 


where N, = m; + n;. 


Let X,,..., X, be independently norm: th ki i 
and meant E(X) = £,, and consider any linear hypothesis with s < n (instead 
of s <n which is required when the variance is unknown). This remains 
invariant under a subgroup of that employed when the variance was unknown, 


and the UMP invariant test has rejection region 


(112) E(x-ÉÓy-ExG-)-IX(-D os 


ally distributed with known variance oj 


with C determined by 


(113) [xowv-e 


i i i i X,,..., Xa) and (Y, E) 
Consider two experiments with observations Qo Mn ] ; 
respectively, where the X, ko i bs FE a ERED A 

2 = 1 and means E(X;) = cju; A a ^ 5 
the Y, poc lomai than that based on the X, if and only if |c;| < 1 for 
i 


all i. 
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[If 1/c2 — 1-- d; with d,> 0, let ¥/ = Y, + V, where V, is N(0, d;) and 
independent of Y, Then c,Y/ has the same distribution as X;. Conversely, if 
c,» 1, the UMP unbiased test of H:6,— 0 against 8,» 0 based on 
(X,,.... X,) is more powerful than the corresponding test based on 
(Qo X) 


Under the assumptions of the preceding problem suppose that E(X,) = $; = 
Lj 14; ;9, E(Y) = n; = 35.50, with the n x s matrices A = (a;;) and 
B = (6,,) of rank s. Then the experiment based on the Y, is more informative 
than that based on the X, if and only if B'B — A’A is nonnegative definite. 
[There exists a nonsingular matrix F such that F’A’AF = I and F'B'BF = A, 
where J is the identity and A is diagonal. The transformation X’ = FX, 
Y' = FY reduces the situation to that of Problem 11.] 

Note. The results of Problems 11 and 12 no longer hold when o? is unknown. 
See Hansen and Torgersen (1974). 


Section 3 


If the variables X,, (j ^ 1,..., nj i — 1,...,s) are independently distributed 
as N(p;, 97), then 


E[Y(x.- x] = G- 08 + Enu- 2X. 


E[E E(X- XJ] = (0-50. 


Let Z;,...,Z, be independently distributed as N({,, a7), i = 1,..., s, where 
the a; are known constants. 


(i) With respect to a suitable group of linear transformations there exists a 
UMP invariant test of H: fj = --- =, given by the rejection region 
(21). 

(i) The power of this test is the integral from C to oo of the noncentral 
x^-density with s — 1 degrees of freedom and noncentrality parameter X 
obtained by substituting {; for Z, in the left-hand side of (21). 


(i) If X,,..., X, is a sample from a Poisson distribution with mean E(X;) 
= À, then yn (VX — VA) tends in law to N(0,1) as n > oc. 
Gi) If X has the binomial distribution b(p, n), then v[arcsin/X/n — 
arcsinyp ] tends in law to N(0,1) as n > oo. 
(Gi) I£(X;, Yj),...,(X,, Yn) is a sample from a bivariate normal distribution, 
then as n — o (in the notation of Chapter 5, Section 15) 


1+R l+p 


— log —- | > N(0,4). 


1 
aR T 
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17. 


18. 
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20. 


21. 


22. 
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Note. Certain refinements of these transformations are discussed by 
Anscombe (1948), Freeman and Tukey (1950), and Hotelling (1953). 
Transformations of data to achieve approximately a normal linear model 
are considered by Box and Cox (1964); for later developments stemming 
from this work see Bickel and Doksum (1981), Box and Cox (1982), and 
Hinkley and Runger (1984). 


. 


` Section 4 
Show that 
r+1 Y+- +Y 2 T Ya de dre aa 
1 r*l H r 
y- ee ii liye a) 0. 
x(" rtl | DU r | = 


(i) For the validity of Theorem 1 it is only required that the probability of 
rejecting homogeneity of any set containing {His Bu) as a proper 
subset tends to 1 as the distance between the different groups (26) all 
> oo, with the analogous condition holding for H3,..., Ay: 

(ii) The condition of part (i) is satisfied for example if homogeneity of a set S 
is rejected for large values of £|X,.— X.J, where the sum extends over the 
subscripts i for which p; € S. 


In Lemma 1, show that a,_, = a is necessary for admissibility. 
Prove Lemma 2 when s is odd. 


Show that the Tukey levels (vi) satisfy (29) when s is even but not when 5 is 
odd. 


The Tukey T-method leads to the simultaneous confidence intervals 


cs 


(114) \(%- X,.) — (y= n) s Ysn(n = 1) 


[The probability of (114) is independent of the p’s and hence equal to 


for all i, j. 


1 - a,] 


Section 6 

The linear-hypothesis test of the hypothesis of no interaction in a two^way 
layout with m observations per cell is given by (39). 

6 with a= b = 2, denote the first three terms 
i iti - 2 $2. and Sjj, corresponding to 
in the partition of ZEX(X;j. X! by Sj, Sp, and Si» 

the A, B, and AB effects (ie. the A's, B's, and y's), and denote by H4, Hs, 
and H,, the hypotheses of these effects being zero. Define a new two-level 
factor B’ which is at level 1 when A and B are both at level 1 or both at level 


In the two-way layout of Section 
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2, and which is at level 2 when A and B are at different levels. Then 
Hy = Hag, Sp = Sap, Hyy = Hg, Sip = Sp, 
so that the B-effect has become an interaction, and the AB-interaction the 
effect of the factor B’. [Shaffer (1977b).] 
The size of each of the following tests is robust against nonnormality: 


(i) the test (35) as b > co, 
(i) the test (37) as mb > oo, 
(iii) the test (39) as m > oo. 


: Note. Nonrobustness against inequality of variances is discussed in Brown 


25. 


26. 


27. 


and Forsythe (19742). 


Let X, denote a random variable distributed as noncentral X? with f degrees 
of freedom and noncentrality parameter X. Then Xy is stochastically larger 
than X, if A < X. 

[It is enough to show that if Y is distributed as N(0,1), then (Y + A^? is 
stochastically larger than (Y + À)?. The equivalent fact that for any z > 0, 


P(IY + A| <z} s P(IY - A| x z), 


is an immediate consequence of the shape of the normal density function. An 
alternative proof is obtained by combining Problem 4 with Lemma 2 of 
Chapter 3.] 


Let X, (i= 1,...,a; je 1... b; k -1,..., m) be independently normally 
distributed with common variance o? and mean 


E(Xj)*n*a*B*yw (La = E&-Yx-0). 
Determine the linear hypothesis test for testing H: a = --- = a, = 0. 


In the three-factor situation of the preceding problem, suppose that a — b — m. 
The hypothesis H can then be tested on the basis of m? observations as 
follows. At each pair of levels (i, j) of the first two factors one observation is 
taken, to which we refer as being in the ith row and the jth column. If the 
levels of the third factor are chosen in such a way that each of them occurs 
once and only once in each row and column, the experimental design is a Latin 
square. The m? observations are denoted by X, jix)» where the third subscript 
indicates the level of the third factor when the first two are at levels i and j.lt 
is assumed that E(X, x) = bia) = + a; + B, + Yx, with Da, = Ef; = Ev. 


(i) The parameters are determined from the £'s through the equations 


fi) = b+ a, fy B+ B. | Saye yi, s eee) = 
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28. 


29. 


30. 


(Summation over j with i held fixed automatically causes summation 
also over k.) 


(ii) The least-squares estimates of the parameters may be obtained from the 
identity 


LE [xm - tol 
ESj 


= my [xiy Aac 7 «E * mY [x49 7 e A] 
*mY[ xq) 7 X7 WP nx, e? 
+E E [iuw 33.04) RT «7t x. 
i 


(iii) For testing the hypothesis H: a, = -> 7a, = 0, the test statistic W* 
of (15) is 


2 
mY [Xie — X3] 
E L [Xo = Xo T Xj 7X + 2X4] /(m - 2) 


The degrees of freedom are m — 1 for the numerator and (m — 1)(m — 2) 
for the denominator, and the noncentrality parameter is V? = mXaj/a*. 
Section 7 

In a regression situation, suppose that the observed values X, and Y, of the 
independent and dependent variable differ from certain true values X/ and Y/ 
by errors U;, V, which are independently normally distributed with zero means 
and variances oj and oj. The true values are assumed to satisfy a linear 
relation: Y/ = a + BXj. However, the variables which are being controlled, 
and which are therefore constants, are the X, rather than the X7. Writing x, 
for X, we have x, = X + U, Y= Y + V, and hence Y, a + Bx, + W, 
where W, = V, — BU. The results of Section 7 can now be applied to test that 
B or a + Bxg have a specified value. 
Let X,,..., Xy; Yu Yn be independently normally distributed with com- 
mon variance o? and means E(X;) = a + B(u, — i), E(Y) = y + (v, ~ 0), 
where the u's and v's are known numbers. Determine the ÜMP invariant tests 
of the linear hypotheses H : f = 8 and H:a-y, $= ô. 
normally distributed with common variance 
c? and means £, = a + Bt; + yt, where the t, are known. If the coefficient 
vectors (th,..-5 t4), k = 0,1,2, are linearly independent, the parameter space 
IIg has dimension s = 3, and the least-squares estimates â, B, y are the 


Let X,,..., X, be independently 
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unique solutions of the system of equations 
aLi + Bg + ye = DX, (k= 0,1,2). 


The solutions are linear functions of the X’s, and if y = Lc; X,, the hypothesis 


y = 0 is rejected when 
mac 
E(X- à- Br - $2) /(m - 3) 


Section 8 
Verify the claims made in Example 8. 


> G6. 


Let Xj, (k ^1,...,nj; i—1,...,a; j ^ 1..., b) be independently nor- 
mally distributed with mean E(X,,,) = £,, and variance o?. Then the test of 
any linear hypothesis concerning the ,, has a robust level provided n,, > 00 
for all i and j. 


In the two-way layout of the preceding problem give examples of submodels 
Ilf) and II? of dimensions s, and s;, both less than ab, such that in one 
case the condition (56) continues to require nj; > œ for all i and j but 
becomes a weaker requirement in the other case. 


Suppose (56) holds for some particular sequence TI” with fixed s. Then it 
holds for any sequence II") C II?) of dimension s’ < s. 

[If Ig is spanned by the s columns of A, let II} be spanned by the first s’ 
columns of A.] 


Let (c,) and {c} be two increasing sequences of constants such that 

€,/C, > 1 as n - co. Then (c, ) satisfies (56) if and only if ( c/) does. 

Let c, = ug + un  ---  u,n*, u; > 0 for all i. Then c, satisfies (56). 

[Apply Problem 35 with c/ = n*] 

(i) Under the assumptions of Problem 30, express the condition (56) in terms 
of the t's. 

(ii) Determine whether the condition of part (i) is equivalent to (51). 

If £, = a + Bt,  yu;, express the condition (56) in terms of the ('s and u’s. 

Show that £}, II? = s, 

[Since the II,, are independent of A, take A to be orthogonal.] 

Show how to weaken (56) if a robustness condition is required only for testing 

a particular subspace II, of Ig. 

[Suppose that II, is given by B, = --- = B, = 0, and use (54).] 

Give an example of an analysis of covariance (46) in which (56) does not hold 


but the level of the F-test of H: a, = ... = a, is robust against nonnor- 
mality. 
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42. 


43. 


45. 


46. 


47. 


Section 9 


(i) A function L satisfies the first equation of (62) for all u, x, and 
orthogonal transformations Q if and only if it depends on u and x only 
through u'x, x'x, and u'u. 

(ii) A function L is equivariant under G, if and only if it satisfies (64). 


(i) For the confidence sets (70), equivariance under G, and G, reduces to 
(71) and (72) respectively. 

(i) For fixed (),,..., y.) the statements Xu, y, € A hold for all (u,,..., u,) 
with Xu? = 1 if and only if A contains the interval (y) =[— yEy? , 


+ Eyl 


(iii) Show that the statement following (74) ceases to hold when r = 1. 
Let X, (i = 1,..., r) be independent N(£,,1). 


(i The only simultaneous confidence intervals equivariant under Go are 
those given by (80). 
(ii) The inequalities (80) and (82) are equivalent. 
(ii) Compared with the Scheffé intervals (69), the intervals (82) for Lu,£) are 
shorter when Lu,é, = £, and longer when uy = +- =u, 


[(ii): For a fixed u = (tys... u,), Eu;y; is maximized subject to |y;| < A for 
all i, by y, = A when u; > 0 and y, = —A when u, < 0] 


Section 10 


G) The confidence intervals L(u; y, $) = Euy; — c(S) are equivariant un- 
der G; if and only if L(u; by, bS) = bL(u; y, S) for all b > 0. 

(i) The most general confidence sets (87) which are equivariant under G;, 
Gn, and G, are of the form (88). 


(i) In Example 11, the set of linear functions Xw;a; = Xw(£,.— £.) for all w 

can also be represented as the set of functions Xw,£,. for all w satisfying 
Ew = 0. 

(i) The set of linear functions LZ); = YXw,(;— £j. £j. £...) for 
all w is equivalent to the set YXw,£;;. for all w satisfying E,w,, = Lj), 
= 0. 

(iii) Determine the simultaneous confidence intervals (89) for the set of linear 
functions of part (ii). 


G) In Example 10, the simultaneous confidence intervals (89) reduce to (93). 
intervals of Example 10 if the 


ii) What change is needed in the confidence in i 
vn v's scole to satisfy (92), i.e. if simultaneous confidence inter- 
vals are desired for all linear functions Xv£, instead of all contrasts? 
Make a table showing the effect of this change for s = 2,3,4,5; n," n = 


3,5,10. 
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48. 


49. 


50. 
51. 


52. 
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Section 11 
G) The test (97) of H: A < A, is UMP unbiased. 


(ii) Determine the UMP unbiased test of H: A = Ap and the associated 
uniformly most accurate unbiased confidence sets for A. 


In the model (95), the correlation coefficient p between two observations 
X,;, Xi, belonging to the same class, the so-called intraclass correlation coeffi- 
cient, is given by p = 03 /(aj + 0°). 

Section 12 
The tests (102) and (103) are UMP unbiased. 
If X, is given by (95) but the number n, of observations per batch is not 
constant, obtain a canonical form corresponding to (96) by letting Y; 


- yn. X,.. Note that the set of sufficient statistics has more components than 
when n, is constant. 


The general nested classification with a constant number of observations per 
cell, under model II, has the structure 
Ae m Bt Apt Bt Gu tU. 


Pm naruto] omo te ob; e miam 2, Cire) i 


(i) This can be reduced to a canonical form generalizing (101). 
(i) There exist UMP unbiased tests of the hypotheses 


H, E 
^'cd...ogd..abk 0 
2 


H MSS. <A 
aT os 


ants the model II analogue of the two-way layout of Section 6, according 
to whi 


(115) X= pt A, t+ B+ Cit Bip 

Ua noge b; ke1,..,n), 
where the 4,, B,, C,j, and E,, are independently normally distributed with 
mean zero and with variances 97; o2, o2, and o? respectively. Determine tests 


which are UMP among all tests that are invariant (under a suitable group) and 
unbiased of the hypotheses that the following ratios do not exceed a given 
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54. 


55. 


56. 


constant (which may be zero): 


6). 02/07; 
(ii) o2/(noà + 0°); 
(iii) of /(nog + 07). 


Note that the test of (i) requires n > 1, but those of (ii) and (iii) do not. 

[Let S} = nbE(X,.— X...)?, Sg = naX(X.,.— X...)?, Sd = nYX(X,.— X,..— 
X, X...)?, S? 2 YXE(X;, — Xij)’, and make a transformation to new 
variables Z,;, (independent, normal, and with mean zero except when i = j = 
k — 1) such that 


a b a b 
S? - Y Zün Si-XY Za» =L X Zh 
i=? j=2 i=2 j=2 


S= y Y Y Z 


i=l jal k=2 


Consider the mixed model obtained from (115) by replacing the random 
variables A, by unknown constants a; satisfying La, = 0. With (ii) replaced by 
(ii^) La? /(noz + o), there again exist tests which are UMP among all tests 
that are invariant and unbiased, and in cases (i) and (iii) these coincide with 
the corresponding tests of Problem 53. 


Consider the following generalization of the univariate linear model of Section 
1. The variables X, (i = 1,..., n) are given by X, = $; + Uj, where (0... Ur) 
have a joint density which is spherical, that is, a function of X7. ,u;, say 


fU- 0) = (EU). 
and the hypothesis H are as in Section 1. 


(i) The orthogonal transformation (1) reduces (X,,..., X,) to canonical 
variables (Y,,...,YX,) with Y, =n; + V» where 9-70 for i=s+ 
1,...,n, H reduces to (3), and the V's have joint density q(0, ++ Yn» 

Gi) In the canonical form of (i), the problem is invariant under the groups Gp 
G;, and G; of Section 1, and the statistic W* given by (7) is maximal 
invariant. 


Under the assumptions of the preceding prob! 
is independent of q and hence the same as in 
r and n — s degrees of freedom. 
[See Chapter 5, Problem 24]. * 5 
Note. The analogous multivariate problem is treated by Kariya (1981), who 


also shows that the test (9) of Chapter 8 continues to be UMP invariant 


The parameter spaces Ig and II, 


lem, the null distribution of w* 
the normal case, namely, F with 


57. 


58. 


59. 


60. 


61. 
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provided q is a nonincreasing convex function. The same method shows that 
this conclusion holds under the same conditions also in the present case. For a 
review of work on spherically and elliptically symmetric distributions, see 
Chmielewski (1981). 


Additional Problems 
Consider the additive random-effects model 


Xj, = b+ A + Btu, (i-1..a; j=l,...,b; k-1,...n), 


where the A's, B's, and U's are independent normal with zero means and 
variances of, oj, and o? respectively. Determine 


(i) the joint density of the X's, 
(i) the UMP unbiased test of H : 03/0? < 8. 


For the mixed model 
X;-7ptatBtU, (i-1..,a; j= l... n), 


where the B’s and u’s are as in Problem 57 and the a’s are constants adding 
to zero, determine (with respect to a suitable group leaving the problem 
invariant) 


(i) a UMP invariant test of H: a, = --- =a,; 
(ii) a UMP invariant test of H: $, = --- =§ =0 ($= p+ a); 


(iii) a test of H:05/o? < 8 which is both UMP invariant and UMP unbi- 
ased. 


Let (X. Los Xj) j=1,...,n, bea sample from a p-variate normal distribu- 
tion with mean (£,,..., £,) and covariance matrix X = (9,;) where oj = o? 
when j= i, and o,,= po? when j * i. Show that the covariance matrix is 
positive definite if and only if p > —1/(p — 1). 

[For fixed c and p « 0, the quadratic form (17e? JEEo,, y, y, = Ly? + PLE; 
takes on its minimum value over X; y? = 1 when all the y's are equal.] 


Under the assumptions of the preceding problem, determine the UMP in- 
variant test (with respect to a suitable G) of H: $, = --- = £y: 

[Show that this model agrees with that of Problem 58 if p = o2/(o2 + 9^) 
v that instead of being positive, p now only needs to satisfy p > — 1/(7 


Permitting interactions in the model of Problem 57 leads to the model 


Xy = Bt A+ B+ Oj + Uy (i-1,...,a; j «1, by k= 1. n) 
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where the A's, B's, C's, and U’s are independent normal with mean zero and 
variances o}, of, a2, and o°. 
(i) Give an example of a situation in which such a model might be 
appropriate. 
(ii) Reduce the model to a convenient canonical form along the lines of 
Sections 5 and 8. 
(ii) Determine UMP unbiased tests of (a) H, : og = 0; (b) Hy: o? = 0. 


Formal analogy with the model of Problem 61 suggests the mixed model 


X, 


Ü 


j= H+ a; tB *CG tU; 


with the B's, C's, and U's as in Problem 61. Reduce this model to a canonical 
form involving X... and the sums of squares 


EG3X. ru - DI C2) 


; 
nog + o° ’ anoj + nop o? 


YXGOGQ-X—-X4XJ LEE Mie = Mm Xt S 


" 
noè + o° o 


Among all tests that are both unbiased and invariant under suitable groups 
under the assumptions of Problem 62, there exist UMP tests of 


ü Ayia = ++: =a,=9; 
(ii) Hy: 02/(nog + 07) < C; 


AR 62 and 63 often are not 
i assumptions of Problems 62 an often 
re Boe P more basic assumptions, sec 


realistic. For alternative models, derived from ¢ 
Scheffé (1956, 1959). Relations between the two types of models are a 
in Hocking (1973), Cohen and Miller (1976), and Kendall, Stuart, and Or 
(1983). ^ 
zu su Xu J= less b bea 
Let (X, 95.20) Kiget Xm or jin s. jn 
ied un-ariate normal distribution. Let EX.) = €i, and denote by 
I.. the matrix of covariances of (Xiro Xij) with (Hia Xon): up: 
pose that for all i, the diagonal elements of 3,, are = r^ and me E ago 
elements = p,7?, and that for i # i^ all n? elements of Z,, are = pT. 


(i) Find necessary and sufficient conditions on p; and p; for the overall 
abn X abn covariance matrix to be positive definite. : i 
Gi) Show that this model agrees with that of Problem 62 for suitable values o 


p, and pz- 
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Tukey's T-Method. Let X, (i= 1,...,r) be independent N(£,,1), and con- 
sider simultaneous confidence intervals 


(6  L[(G,j))5x]s&-& «M[(, x] foral ij. 


The problem of determining such confidence intervals remains invariant under 
the group Gj of all permutations of the X's and under the group G, of 
translations gx = x + a. 


(i) In analogy with (61), attention can be restricted to confidence bounds 
satisfying 
(117) L[G, j); x] = -M[G. i); x]. 


(i) The only simultaneous confidence intervals satisfying (117) and equi- 
variant under Gj and G, are those of the form 


(118) S(x) -(£ix-x-A^«£&-&«x-x + A forall i +j}. 
Gii) The constant A for which (118) has probability y is determined by 
(119) P{ max |X, — Xi « A} = (X, Xa < A} =y, 


where the probability P; is calculated under the assumption that £j = 
e =g. 


In the preceding problem consider arbitrary contrasts Ec,¢,, Ec, = 0. The 
event 


(120) x,-x)-(&-£&)s^ forall i*j 


is equivalent to the event 


A * 
(121) IZ«x - Ee&|s7 Ell forallc with c, = 0, 


which therefore also has probability y. This shows how to extend the Tukey 
intervals for all pairs to all contrasts. 

[That (121) implies (120) is obvious. To see that (120) implies (121), let 
J; = x, — §; and maximize |Y.c, y,| subject to ly; 7 4 < å for all i and j. Let 
P and N denote the sets (i: c, > 0) and {i: c; < 0), so that 


Lay= Ean- Y. laly 


ieP ieN 


Then for fixed c, the sum Ec; y, is maximized by maximizing the y,'s for i € P 
and minimizing those for i € N. Since ly; — y| < A, it is seen that Ec, y, iS 
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maximized by y; = 4/2 for i € P, y, = —A/2 for i € N. The minimization of 
Lc; y, is handled analogously.] 


6. G) Let X, (j *l..., n; i=1,...,s) be independent N(£, o?), o? un- 
known. Then the problem of obtaining simultaneous confidence intervals 
for all differences £,— £, is invariant under Gj, G,, and the scale 
changes G;. 

(ii) The only equivariant confidence bounds based on the sufficient statistics 
X,. and S? = XE(X,; — X)? and satisfying the condition corresponding 
to (117) are those given by 


N 
(122) so) = [eis GS SHH 
XY 
Sx. Xt S 
n-s 


for all i + j j 
with A determined by the null distribution of the Studentized range 


max |X.— Xi 2 s) .j* 
(123) Py $/ ES 
Gii) Extend the results of Problem 66 to the present situation. 
68. Construct an example [i.e., choose values m = «°° = n =n and « and a 


particular contrast (¢),--+» c,)] for which the Tukey confidence intervals det 
are shorter than the Scheffé intervals (93), and an example in which the 


situation is reversed. 
.,m) and Xj (= 1,...5 Em 


69. s | Ley (eb 
Dea Mee on a standard and s competing new € 
ments and suppose the X's are independently distributed as N(£o, 0^) a 
N(£,, o?) respectively. ze Problems 65 and 67 to the problem o 


obtaining simultaneous confidence intervals for the s differences & 7 £o 


(i =1,...,5). 
70. In generalization of Problem 66, show how to extend the Dunnett intervals of 
Problem Sie € A for i= 1,...,5 is equivalent to the 


event |y; — Jo 1... s 15 
[Use the fact that the ves all (6o. c) satisfying Ei = 0] 


8, the intervals resulting 
d Dunnett) methods to all contrasts are 
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were designed and for contrasts close to them, and longer for some other 
contrasts. For details and generalizations, see for example Miller (1981), 
Richmond (1982), and Shaffer (19772). 


71. In the regression model of Problem 8, generalize the confidence bands of 
Example 12 to the regression surfaces 


O hle... e,)= Y -i6;Bj; 
(ii) hj(e5,..., es) = By + 3-16). 
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CHAPTERS 


Multivariate Linear Hypotheses 


1. A CANONICAL FORM 


The univariate linear models of the preceding chapter arise in the study of 
the effects of various experimental conditions (factors) on a single character- 
istic such as yield, weight, length of life, or blood pressure. This characteris- 
tic is assumed to be normally distributed with a mean which depends on the 
various factors under investigation, and a variance which is independent of 
these factors. We shall now consider the multivariate analogue of this 
model, which is appropriate when one is concerned with the effect of one or 
more factors simultaneously on several characteristics, for example the effect 
of a change in the diet of dairy cows on both fat content and quantity of 
milk. 

The multivariate generalization of a real-valued normally distributed 
random variable is a random vector (X,,..., X) with the multivariate 
normal probability density 


(1) A el-el 3 £)(x,- £)]. 


where the matrix A = (a, j) is positive definite, and |A| denotes its determi- 
nant. The means and covariance matrix of the X's are given by 


Q0) r(x)e&, E- EH E) o ()74^ 


Consider now n independent multivariate normal vectors X, = 
Co X, ig = Moe with means E( X,;) = ai and rura co- 
Variance matrix A~} As in the univariate case, a D 
hypothesis is defined in terms of two linear subspaces Ilg d " T 
n-dimensional space having dimensions s <n and 0$ s — 7 s s. He 
assumed known that for all i = 1,..., p, the vectors (CER Eni) a g 
the hypothesis to be tested specifies that they lie in IL,. This problem 
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reduced to canonical form by applying to each of the p vectors ( Xs;, ..., Xni) 
the orthogonal transformation (1) of Chapter 7. If 


and the transformed variables are denoted by X5, the transformation may 
be written in matrix form as 


X* = CX, 


where C = (cag) is an orthogonal matrix. 

To obtain the joint distribution of the X consider first the covariance of 
any two of them, say X = X7 .,c,, X,, and Xf = 15-105, Xa; Using the 
fact that the covariance of X,, and X,, is zero when y + 6 and cj; when 
y = 9, we have 


n 


n 
Cov( X, X£) = Y E cont aCov( X,, Xy) 
garðs 


Y 9j when a= f, 
tee g i when a x B. 


The rows of X* are therefore again independent multivariate normal vectors 
with common covariance matrix 4-!. It follows as in the univariate case 
that the vectors of means satisfy 


j=: 45$-50 (i=1,..., p) 
under Q, and that the hypothesis becomes 
H: &- A -£5-0 (1:15... p). 


Changing notation so that Y 's, U’s, and Z's denote the first r, the next 
s — r, and the last m — n — s sample vectors, we therefore arrive at the 
following canonical form. The vectors Y,, Ug, Z, (a = L..., r5 B=1,.-+ 
s — r; y= L..., m) are independently distributed according to p-variate 
normal distributions with common covariance matrix 4 '. The means 9 
the Z's are given to be zero, and the hypothesis H is to be tested that the 
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means of the Y's are zero. If 


Yuan Zu Zip 
X MS : | and Z=] : pl 
Laon thie da Zm E y 


invariance and sufficiency will be shown below to reduce the observations to 
the p X p matrices Y'Y and Z'Z. It will then be convenient to have an 
expression of these statistics in terms of the original observations. 

As in the univariate case, let (£j, .., £,,) and (Ẹni: -+ Eni) denote the 
projections of the vector (X; ..., Xni) on Ig and IL. Then 


is the inner product of two vectors, each of which is the difference between a 
given vector and its projection on Igo. It follows that this quantity is 
unchanged under orthogonal transformations of the coordinate system in 
which the variables are expressed. Now the transformation 


Xi 


i i i oordi- 
may be interpreted as expressing the vector (Xi ED X,;) in a new coor 
nate system, the first s coordinate axes of which lie in IIg. The projection 


on II ed vector Yin.. Yo Up Uno Zi Zm 
ue 0. Ha so that the difference between the 


is (Y. Yop Ue sen Crema F , 
vector andis projection is (0,..-0, Zyp-.-, Za). The ijth element of ZZ 
is therefore given by 
m E ê 
(3) X ZuZyj i » (X. 7 &)( T fn): 
a-1 


Analogously, the projection of the word A Qnid 

U,... T o Ooa Di tl 

0,:..,0) and the difference between the projections on Ig iae n. : 

therefore (Y, y..0,...,0,..., 0). It follows that the sum 5. iYgiYgj 
ays soos CUIDA 


equal to the inner product (for the ith and jth vector) of the d he 
these projections. On comparing this sum with the expression © 
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inner product in the original coordinate system, it is seen that the ijth 
element of Y'Y is given by 


n 


(4) E YgYs; FP. 5 (£; x ££; T EN. 


a=1 


2. REDUCTION BY INVARIANCE 


The multivariate linear hypothesis, described in the preceding section in 
canonical form, remains invariant under certain groups of transformations. 
To obtain maximal invariants under these groups we require, in addition to 
some of the standard theorems concerning quadratic forms, the following 
lemma. 


Lemma 1. Jf M is any m X p matrix, then 

(i) M’M is positive semidefinite, 

(ii) the rank of M'M equals the rank of M, so that in particular M'M is 
nonsingular if and only if m 2 p and M is of rank p. 


Proof. (i): Consider the quadratic form Q = u'(M'M)u. If w = Mu, 
then 


Q — w'w 2 0. 


(ii): The sum of squares w^w is zero if and only if the vector w is zero, 
and the result follows from the fact that the solutions u of the system of 
equations Mu — 0 form a linear space of dimension p — p, where p is the 
rank of M. 


We shall now consider three groups under which the problem remains 
invariant. 


G,. Addition of an arbitrary constant dg, to each of the variables Up; 
leaves the problem invariant, and this eliminates the U’s, since the Y’s and 
Z’s are maximal invariant under this group. 


G,. In the process of reducing the problem to canonical form it was 
seen that an orthogonal transformation 


Yy* = CY 


affects neither the independence of the row vectors of Y nor the covariance 
matrix of these vectors. The means of the Y*'s are zero if and only if those 
of the Y's are, and hence the problem remains invariant under these 
transformations. 


8.2] REDUCTION BY INVARIANCE 457 


The matrix Y'Y of inner products of the column vectors of Y is invariant 
under G,, since Y*Y* = Y'C'CY = Y'Y. The matrix Y'Y will be proved to 
be maximal invariant by showing that Y'Y — Y*'Y* implies the existence of 
an orthogonal matrix C such that Y* = CY. Consider first the case r = p. 
Without loss of generality the p column vectors of Y can be assumed to be 
linearly independent, since the exceptional set of Y's for which this does 
not hold has measure zero. The equality Y'Y = Y*'Y* implies that C = 
Y*Y-! is orthogonal and that Y* = CY, as was to be proved. Suppose next 
that r > p. There is again no loss of generality in assuming the p column 
vectors of Y to be linearly independent. Since for any two p-dimensional 
subspaces of r-space there exists an orthogonal transformation taking one 
into the other, it can be assumed that (after a suitable orthogonal transfor- 
mation) the p column vectors of Y and Y* lie in the same p-space, and the 
problem is therefore reduced to the case r — p. If finally r < p, the first 7 
column vectors of Y can be assumed to be linearly independent. Denoting 
the matrices formed by the first r and last p — r columns of Y by Y, and 
Y,, so that 


y=(% X) 


one has Y,"Y,* = Y/Y,, and by the previous argument there exists an 
orthogonal matrix B such that Y;* = BY,. From the relation YY = YY 
it now follows that Y = (Y) "Y/Y, = BY;, and this completes the 


proof. 
Similarly the problem remains invariant under the orthogonal transfor- 
mations 


Z* = DZ, 


al invariant, Alternatively the reduction to Z'Z 
Z'Z together with the Y 's and U's aA 
set of sufficient statistics. In either case the problem ae the groups G, 
and G, reduces to the two matrices V = Y'Y and $ = ZZ. 

i icti Problem 1), which 
G,. We now impose the restriction m = p (see Pro 

assures that there are enough degrees of freedom to Lore o a 
estimate of the covariance matrix, and consider the transform. 


Which leave Z'Z as maxim: 
can be argued from the fact that 


y* = YB, Z* = ZB, 


i i ix. These transformations act 
wher nsingular p x p matrix. These 
M independent multivariate normal vectors 


the problem invariant. The 
(Ya, ..., Yap)» (Zye Zip and clearly leave the pi 


458 MULTIVARIATE LINEAR HYPOTHESES [8.2 
induced transformation in the space of V = Y'Y and S = Z'Z is 
V* — B'VB, * — B'SB. 


Since |B'(V — AS)B| = |B?|V — AS}, the roots of the determinantal equa- 
tion 


(5) IV —AS|-0 


are invariant under this group. To see that they are maximal invariant, 
suppose that the equations |V — AS|=0 and |V* — AS*| = O have the 
same roots. One may again without loss of generality restrict attention to 
the case that p of the row vectors of Z are linearly independent, so that the 
matrix Z has rank p, and that the same is true of Z*. The matrix S is then 
positive definite by Lemma 1, and it follows from the theory of the 
simultaneous reduction to diagonal form of two quadratic forms! that there 
exists a nonsingular matrix B, such that 


BYB,=A, — B(SB, = I, 


where A is a diagonal matrix whose elements are the roots of (5) and T is 
the identity matrix. There also exists B, such that 


BIV*B,— A, — BIS*B, =I, 
and thus B = B,B;! transforms V into V* and S into S*. 


Of the roots of (5), which constitute a maximal set of invariants, some 
may be zero. In fact, since these roots are the diagonal elements of ^, the 
number of nonzero roots is equal to the rank of A and hence to the rank of 
V = B( AB; |, which by Lemma 1 is min(p, z). When this number is 
> 1, a UMP invariant test does not exist. The case p = 1 is that of a 
univariate linear hypothesis treated in Section 1 of Chapter 7. We shall now 
consider the remaining possibility that r = 1. 

When r = 1, the equation (5), and hence the equivalent equation 


|VS-! — AI] =0, 
has only one nonzero root. All coefficients of powers of A of degree 


«p- 1 therefore vanish in the expression of the determinant as a poly- 
nomial in À, and the equation becomes 


(A)? + W(-A)" =0, 


+See for example Anderson (1984, Appendix A, Theorem A.2.2). 
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where W is the sum of the diagonal elements (trace) of VS~1. If S" denotes 
the ijth element of S^! and the single Y-vector is (Y,,..., Y,), an easy 
computation shows that 


PP 
(6) w= Xs. 
i=1 j=1 
A necessary and sufficient condition for a test to be invariant under G;, G, 
and G, is therefore that it depends only on W. 
The distribution of W depends only on the maximal invariant in the 
parameter space; this is found to be 


Pi op 
(7) y = X X 4; NiNj> 
i=l j=1 


where n; = E(Y,), and the probability density of W is given by (Problems 
5-7) 


k 1 
"E (142) wip-i*k 
o aora S 


This is the same as the density of the test statistic in the univariate case, 
given as (6) of Chapter 7, with r — p and n-semtl-p For any 
bo < V, the ratio p, (w)/py(w) is an increasing function of w, and it 
follows from the Neyman-Pearson lemma that the most powerful invariant 
0 rejects when W is too large, or 


test for testing H: m = u == 
equivalently when 
cdd 
(9) Duro ys 
ie 


i i f Student's (, is 
The quantity mW, which for p — 1 reduces to the square o 

essentially Hotelling’s T2-statistic, to which it specializes in the iir ied 
test to be considered in the next section. The constant C is auod 
the fact that for y = 0 the statistic (m + 1 — p)W/p has the F-dis 


with p and m + 1 — p degrees of freedom. As in the univariate case, there 


also exists a UMP invariant test of the more general hypothesis H”: 
V? < 42, with rejection region W>C’. 
LEMS 


e hypothesis 
is a sample 


3. THE ONE- AND TWO-SAMPLE PROB 


The simplest special case of a linear hypothesis with Sm th 
Hide =€,=0, where as a A aE 
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from a p-variate normal distribution (1) with unknown mean (£,,..., &,), 
covariance matrix © = A~}, and p < n — 1. It is seen from Example 4 of 
Chapter 7 that 


By (3), the ijth element S;; of S = Z'Z is therefore 
$j7 È (X,- X3(X,;- X.)), 


and by (4) 


With these expressions the test statistic is the quantity W of (6), and the test 
is given by (9) with s = 1 and hence with m = n — s = n — 1. The statistic 
T? — (n — 1)W is known as Hotelling's T?. The noncentrality parameter 
(7) in the present case reduces to y? = XX, 73:32 

The test shares the robustness properties of the corresponding univariate 
t-test discussed in Chapter 5, Section 4. Suppose that (X,,..., Xap) is a 
sample from any p-variate distribution F with vector mean zero and finite, 
nonsingular covariance matrix X, and write 


(10) T? = YY n X.(n — 1)S n X.. 


Using the fact that S,/(n — 1) tends in probability to o, j and that 
(Vn X.,,..., Vn. X.,) has a p-variate normal limit distribution with covari- 
ance matrix È, it is seen (Problem 8) that the null distribution of T? tends 
to the x?-distribution as n > oo. Thus, asymptotically the significance level 
of the T?-test is independent of F. However, for small n, the differences 
may be substantial. For details see for example Everitt (1979), Davis (1982), 
Srivastava and Awan (1982), and Seber (1984). 

The T?-test was shown by Stein (1956) to be admissible against the class 
of alternatives 4? » c for any c » 0 by the method of Theorem 8 of 
Chapter 6. Against the class of alternatives y? < c admissibility was proved 
by Kiefer and Schwartz (1965) [see Problem 47, and also Schwartz (1967) 
and (1969)]. 

The problem of testing H against one-sided alternatives such as K : £; > 0 
for all i, with at least one inequality strict, is treated by Perlman (1969) and 
in Barlow et al. (1972), which gives a survey of the literature. Minimal 
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complete classes for this and related problems are discussed by Marden 
(1982). 

Most accurate equivariant confidence sets for the unknown mean vector 
(£,....£,) are obtained from the UMP invariant test of H:£; = £j; 
(i = 1,..., p), which has acceptance region 


nYX QC; = &)(n — 1)9"(X.,- Fo) < C. 
The associated confidence sets are therefore ellipsoids 
(11) nE (E ET X,,)(n EN 1)s"(£, E X.) sc 


centered at (X.,,..., X.,). These confidence sets are equivariant under the 
groups G,—G, of Section 2 (Problem 9), and by Lemma 4 of Chapter 6 are 
therefore uniformly most accurate among all equivariant confidence sets at 
the specified level. : t ^ 2. 

Consider next the two-sample problem in which (Gs Xy aw 
L...,n, and (Xf?,..., X), B — b... n» are independent samples 
from multivariate normal distributions with common covariance matrix 
A^! and means (£(),..., £0) and (£0... EO Suppose that p < nj + 
n, — 2,* and consider the hypothesis H: £f? = £0 for i — 1,:.., p. Then 
5 — 2, and it follows from Example 5 of Chapter 7 that for all a and f 


saxo, EP =x? 
and 


m n; 

Expr EXP 
$ A z B=1 Bio 
ê te a S 
[ae e 5. n +My ! 


Hence 


[] 


ea -xa 
s,- Y (x@ - xoyxg xp) + PAC VG x9), 


a=1 


and the expression for Y,Y, can be simplified to 


ya E - XYx9- xX). 
vx, = n (xt- KYA - X) + (09 = OG ) 


YE 1958). 
^A est of H. torti Dict toms =e Un Deme © ) 
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Since m = n — 2, T? = mW is given by 
(12) T2=n(n—2)(X® — XOys-!(x — x9), 


where n =n, + n, and XM = (XW --- Xy, k = 1,2. 

As in the one-sample problem, this test is robust against nonnormality 
for large n, and n; (Problem 10). In the two-sample case, the robustness 
question arises also with respect to the assumption of equal covariances for 
the two samples. The result here parallels that for the corresponding 
univariate situation: if n,/n, > 1, the asymptotic distribution of T? is the 
same when Z, and >, are unequal as when they are equal; if n/n, > p * lj 
the limit distribution of T? derived for Z, = X, no longer applies when the 
covariances differ (Problem 11). 

Tests of the hypothesis ¢ = £O (i — 1,..., p) when the covariance 
matrices are not assumed to be equal (i.e. for the multivariate Behrens—Fisher 
problem) have been proposed by James (1954) and Yao (1965) and are 
studied further in Subrahmaniam and Subrahmaniam (1973,1975) and 
Johansen (1980). Their results are summarized in Seber (1984). For related 
work, see Dalal (1978), Dalal and Fortini (1982), and Anderson (1984). The 
effect of outliers is studied by Bauer (1981). 

Both the one- and the two-sample problem are examples of multivariate 
linear hypotheses with r equal to 1, so that a UMP invariant test exists and 
is of the T? type (9). Other problems with r= 1 arise in multivariate 
regression (Problem 13) and in some repeated-measurement problems (Sec- 
tion 5). 

Instead of testing the value of a mean vector or the equality of two mean 
vectors in the one- and the two-sample problem respectively, it may be of 
interest to test the corresponding hypotheses 2 = Ej or Z, = X, concern- 
ing the covariance matrices. Since the resulting tests, as in the univariate 
case, are extremely sensitive to the assumption of normality, they are not 
very useful and we shall not consider them here. They are treated from an 
invariance point of view by Arnold (1981) and by Anderson (1984), who 
also discusses more robust alternatives. In the one-sample case, another 
problem of interest is that of testing the hypothesis of independence of two 
sets of components from each other. For the case p — 2, this was considered 
in Chapter 5, Section 13. For general p, see Problem 45. 


4. MULTIVARIATE ANALYSIS OF VARIANCE (MANOVA) 


When the number r of vector constraints imposed by H on a multivariate 
linear model with p > 1 exceeds 1, a UMP invariant test no longer exists. 
Tests based on various functions of the roots À; of (5) have been proposed 
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for this case, among them 


(i) the Lawley-Hotelling trace test, which rejects for large values of 

YA; 

(ii) the likelihood-ratio test (Wilks A), which rejects for small values of 
\V|AV +,S| or equivalently of TI1/(1 + A,) (Problem 18); 

(iii) the Pillai-Bartlett trace test, which rejects for large values of 
EX/Q + Aj); 

(iv) Roy's maximum-root test, which rejects for large values of max ;. 

Since these test statistics are all invariant under the groups G,-G, of 


Section 1, their distribution depends only on the maximal invariants in the 
parameter space, which are the nonzero roots of the equation 


(13) ]B.— A320; 


where X is the common covariance matrix of (X,,..., Xap) and B is the 


p X p matrix with.(i, j)th element 


Y E(&; - L)E(E; Fi fy): 


a=1 
the tests (i)-(iv) are given among 


Olson (1976), and Stevens (1980), 
in the power of (i)-(iii), but 


Some comparisons of the power of 
others by Pillai and Jayachandran i 
and suggest that there is little difference 

i i ith (iv). This last test tends to be more powerfu 
considerable difference with (iV). Menon in which (13) has em 


against alternatives that approximate the sit 
one nonzero root, that is, alternatives in which all but one of the roots are 


close to zero and there is one (positive) root that is widely Wig tom 
the others (see Problem 19 for an example). On the other a b “A 
maximum-root test tends to be less powerful than the other three when 


has several roots which differ considerably from €: ge endo 


i i x l 
asymptotic result. To motivate the asymptotics, consider first the s-sample 
h : X19), am 1... Ap Km hem are sam- 

normal distributions with mean (Eier ep D 


x X. For testing H:£f) = 77 = £(? for all 
j — 1,..., p, the matrices V and S have elements (Problem 16) 


(14) y,- Xn? - xjJ(xt? - x) 
k 
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and 


w^ Hy 


(15) $,- YY (XP = XP)( XH = xP), 
k=l a=1 


where X,, = En X(P/Yn,. Under the hypothesis, the joint distribution of 
the V, is independent of m,,..., 75, while S,;/(n — s) tends in probability 
to the (i, j)th element c;; of 2. 

Analogously, in other analysis-of-variance situations, as the cell sizes 
tend to infinity, the distribution of V under H remains constant while 
S;;/(n — s) tends in probability to o; . 

Let À,,..., À, denote the a = min( p, r) nonzero roots of 


S 
(16) |V - AS| 2 |V — (n — s)À - 0, 
n-—s 
and Af,..., A*, the nonzero roots of 
(17) V- AX| - 0, 


the null distribution of which we suppose to be independent of n. Then it is 
plausible and easy to show (Problem 21) that ((n — s)À,,.... (n — 5)Aq) 
tends in law to (M,..., A*) and hence that the distribution of T, = (1 — 
5), tends to that of XA* as n > oo. If 


T,=(n-s)¥ and T,=(n-—s)log] [(1+A,), 


Y 
[m 


we shall now show that T, — T, and T; — T, tend to zero in probability, so 
that 7, T}, and T, are asymptotically equivalent and in particular have the 
same limit distribution. 


(a) The convergence of the distribution of (n — s)À; implies that 
A, 0 in probability and hence that T, — T, tends to zero in 
probability. 


(b) The expansion log(1 + x) = x[1 + o(1)] as x > 0 gives 
(— s)logTT( A;) = (n- s) Xog(1 - A;) = (n7 5); * Ro» 


where R, > 0 in probability by (a). 
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Thus, the distributions of Tj, T}, and T} all tend to that of Xv. On the 
other hand, the distribution of the normalized maximum-root statistic 
(n — s)max À; tends to the quite different distribution of max t. 

The null distribution of XM is the limit distribution of 7;, T}, and T, 
and therefore provides a first, crude approximation to the distribution of 
these statistics under H. We shall now show that this limit distribution is x? 
with rp degrees of freedom. 

To see this, consider the linear model in its canonical form of Section 1, 
in which the rows of the r X p matrix Y are independent p-variate normal 
with common covariance matrix 2 and mean ņ = E(Y), but where È is 
now assumed to be known. Under H, the matrix 7 is the r X p zero matrix. 
There exists a nonsingular transformation Y* = YB such that the covari- 
ance matrix B'XB of the rows of Y* is the identity matrix. The variables 
Y,* (a =1,...,7; i=1,..., p) are then independent normal with means 
n*, = E(Y,3) and unit variance. The hypothesis becomes H : n, = 0 for all 
a and i, and the UMP invariant test (under orthogonal transformations of 
the pr-dimensional sample space) rejects when YXY? > C. The test statis- 
tic EEY,f? is the trace of the matrix V* = Y*Y* = B'VB and is therefore 
the sum of the roots of the equation |B’VB — AJ| = 0. Since J = BSB, 
they are also the roots of |V — AZ| = 0 and hence LEY,*? = EM, and this 
completes the proof. act 

More accurate approximations, and tables of the null distributions of the 
four tests, are given in Anderson (1984) and Seber (1984). p-values are also 
provided by the standard computer packages. Ahaia 

The robustness against nonnormality of tests for univariate linear hy- 
potheses extends to the joint distribution of the roots À; of (5) as it did for 
the single root in the case r = 1. This is seen by showing that, as (epu 
S, /(n — s) tends in probability to 9,,, and that the joint distribution o ; s 
variables Y, , (i= lcs maji does P) and hence of the elements of 
tends to a limit which is independent of the underlying error MAS. 
(see for example Problems 20 and 21). For more details, see eie: dide d 
Simulation studies by Olson (1974) € ja of the four tests, 

iii) is the most and that of (iv) the least robust. 1 » r 
Dusen of multivariate linear models from a Bayesian ge scil 
can be found, for example, in Box and Tiao (1973), in Press an gi 


(1985), and in the references cited there. 


5. FURTHER APPLICATIONS 


iate li i in the preced- 
The invariant tests of multivariate linear hypotheses ages : E P € 
ing sections apply to the multivariate analogue of any ne pena 
hypothesis, and the extension of the univariate to the corresp 
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variate test is routine. In addition, these tests have applications to some 
hypotheses that are not multivariate linear hypotheses as defined in Section 
1 but which can be brought to this form through suitable transformation 
and reduction. 

In the linear hypotheses of Section 1, the parameter vectors being tested 
are linear combinations 


n n 
Dib Dc EX), ywl,..r 


íi yml 


where the X, are the n independent rows of the observation matrix X. We 
shall now instead consider linear combinations of the corresponding column 
vectors, and thus of the (dependent) components of the p-variate distribu- 
tion. 


Example 1. Let (Xa,---, Xaqs X, 41 Xa,2q)) & = 1,..., n, be a sample 
from a multivariate normal distribution, and consider the problem of testing 
H: £,,, = $; for i — 1,..., q. This might arise for example when X,,,..., Xaq and 
X, gk Xa 2q are q measurements taken on the same subject before and after a 
certain treatment, or on the left and right sides of the subject. 


Example 2. Let (X,,..., X,,), a — 1,...,n, be a sample from a p-variate 
normal distribution, and consider the problem of testing the hypothesis H:£, = 
T am p: As an application suppose that a shop has p machines for manufacturing 
a certain product, the quality of which is measured by a random variable X. In an 
experiment involving n workers, each worker is put on all p machines, with X, 
being the result of the ath worker on the ith machine. If the n workers are 
considered as a random sample from a large population, the vectors (X,,..., Xap) 
may be assumed to be a sample from a p-variate normal distribution. Of the two 
factors involved in this experiment one is fixed (machines) and one random 
(workers), in the sense that a replication of the experiment would employ the same 
machines but a new sample of workers. The hypothesis being tested is that the fixed 
effect is absent. The test in this mixed model is quite different from the correspond- 


ing model I test where both effects are fixed, and which was treated in Section 5 of 
Chapter 7. 


An important feature of such repeated measurement designs is that the p 
component measurements are measured on a common scale, so that it is 
meaningful to compare them. (This is not necessary in the general linear- 
hypothesis situations of the earlier sections, where the comparisons are 
made separately for each fixed component over different groups of subjects.) 
Although both Examples 1 and 2 are concerned with a single multivariate 
sample, this is not a requirement of such designs. Both examples extend for 
instance to the case of several groups of subjects (corresponding to different 
conditions or treatments) on all of which the same comparisons are made 
for each measurement. 
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Quite generally, consider the multivariate linear model of Section 1 in 
which each of the p column vectors of the matrix 


is assumed to lie in a common s-dimensional linear subspace Ilg of 
n-dimensional space. However, the hypothesis H is now different. It specifies 
that each of the row vectors of £ lies in a (p — d)-dimensional subspace IT’, 
of p-space. In Example 1, s = 1, p — d = q; in Example 2, s = p — d — 1. 

As a first step toward a canonical form, make a transformation Y = XE, 
E nonsingular, such that under H the first d columns of 4 = E(Y) are 
equal to zero. This is achieved by any E the last p — d columns of which 
span II’,. The rows of Y are then again independent, normally distributed 
with common covariance matrix, which is now E’ZE. Also, since each 
column of 7 is a linear combination of the columns of the matrix £ — E( X), 
the columns of 7 lie in IIg. If we write 


Y) qs n ia) | 
d p-d 


the matrix 7, under H reduces to the n X d zero matrix. y 
Next, subject Y to an orthogonal transformation CY, with the first s 


rows of C spanning IIg, and denote the resulting matrix by 


(3 ON 
ZV 


d p-d=l 


we 
i) 


(18) C 


hat the rows of (18) are p-variate 


Then it follows from Chapter 7, Section 1 t s 
'SE and with means 


normal with common covariance matrix E 


E(Y)-»  E(2)-0 E(U)=»,  E(V)-0. 


P à inar 
In thi nical form, the hypothesis becomes H:n 
The ‘probe of testing H remains invariant under the group G, of 


adding arbitrary constants to the Is elements of U, which leaves Yi Zi gio x 
as maximal invariants. The next step is to show that invariance conside 


tions also permit the discarding of V. 
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Let G, be the group of transformations 
(19) V* = ZB + VC, El xb. Y 


where B is any d X / and C any nonsingular / X / matrix. Before applying 
the principle of invariance, it will be convenient to reduce the problem by 
sufficiency. The matrix Y together with the matrices of inner products Z'Z, 
V'V, and Z'V form a set of sufficient statistics, and it follows from Theorem 
6 of Chapter 6 that the search for a UMP invariant test can restrict 
attention to these sufficient statistics (Problem 24). We shall now show that 
under the transformations (19), the matrices Y and Z'Z are maximal 
invariant on the basis of Y, Z'Z, V'V, and Z'V. 

To prove this, it is necessary to show that for any given m X / matrix 
V** there exist B and C such that V* = ZB + VC satisfies 


Z'y* = zZrys* and V*y*- yswyws, 


Geometrically, these equations state that there exist vectors (V;*,..., V, 5). 
i=1,...,/ in the space S spanned by the columns of Z and V which have 
a preassigned set of inner products with each other and with the column 
vectors of Z. 

Consider first the case / = 1. If d + 1 > m, one can assume that Z and 
the column of V span S, and one can then take V** = V*. If d -- 1 « m, 
then Z and the column of V may be assumed to be linearly independent. 
There then exists a rotation about the columns of Z as axis, which takes V** 
into a vector lying in S, and this vector has the properties required of V*. 

The proof is now completed by repeated application of the result for this 
special case. It can be applied first to the vector (Viv ..., Vin), to determine 
the first column of B and a number c,, to which one may add zeros to 
construct the first column of C. By adjoining the transformed vector 
(Vi, -.., V4) to the columns of Z and applying the result to the vector 
(Viz ---» V,,;), one obtains a vector (Vi$,..., V5) which lies in the space 
spanned by (Vi, ..., Vm), (V5, ..., V.,;) and the column vectors of Z, and 
which in addition has the preassigned inner products with (V;%,..., V4). 
with the columns of Z and with itself. This second step determines the 
second column of B and two numbers c,;, €; to which zeros can be added 
to provide the second column of C. Proceeding inductively in this way, one 
obtains for C a triangular matrix with zeros below the main diagonal, so 
that C is nonsingular. Since Z, V, and V** can be assumed to have maximal 
rank, it follows from Lemma 1 and the equation V'V* = V**'V** that the 
rank of V* is also maximal, and this completes the proof. 

Thus invariance reduces consideration to the matrices Y and Z, the rows 
of which are independently distributed according to a d-variate normal 
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distribution with common unknown covariance matrix. The expectations are 
E(Y)=n, E(Z)=0, and the hypothesis being tested is H:n = 0, a 
multivariate linear hypothesis with r = s. In particular when s = 1, as was 
the case in Examples 1 and 2, there exists a UMP invariant test based on 
Hotelling’s T?. When s > 1, the tests of Section 4 become applicable. In 
either case, the tests require that m > d. 

In the reduction to canonical form, the p X p matrix E could have been 
restricted to be orthogonal. However, since the covariance matrix of the 
rows is unknown (rather than being proportional to the identity matrix as 
was the case for the columns), this restriction is unnecessary, and for 
applications it is convenient not to impose it. 

It is also worth noting that 


(3) =ch, 


so that (Y, Z) is equivalent to Y,. In terms of (%, Y,) the invariance 
argument thus reduces the data to the maximal invariant Y;. 


Example 1. (continued). For the transformation XE take 


Dj; Xu SUD. Wai = X a-21,.,n, i-71,5.,4q 
i invari the data to the 
By the last remark preceding the example, invariance then reduces à 
nage (D,;); Hien eae previously denoted oy dog Dib: j » ) n gas 
sample from a q-variate normal distribution with mean (01... » 74/5 Ui 264-65: 
The hypothesis H reduces to ô = 0 for all i, and the UMP RAT test is 
Hotelling’s one-sample test discussed in Section 3 (with q in place o p) A 
To illustrate the case s > 1, suppose that the experimental icu consi 
groups, and denote the p = 24 measurements on each subject by 


E AND Xagi APTE AE ac leu 


and 


gio Ms Xe Mza) B=1,. n 
i =¢* for i=1,... q, Which might 
Consi thesis H: fga = En Ep = 8 à 
sie pe EE circomsisies as in the one-sample case. The same argument as 
before now reduces the data to the two samples 


(Dypreves Dag) athens 


and 
(Bios ion io fedis m 
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with means (6,,...,6,) and (8f,..., 87), and the hypothesis being tested becomes 
H:8,= --- =6,=0, 8t = --- =8% =0. This is a multivariate linear hy- 
pothesis with r = s = 2 and p = q, which can be tested by the tests of Section 4. 


A linear hypothesis concerning the row vectors (£,,,..., ap) has been 
seen in this section to be reducible to the linear hypothesis » = 0 on the 
reduced variables Y and Z. To consider the robustness of the resulting tests 
against nonnormality.in the original variables, suppose that X,, = a; + Wai 
where (W,,..., Wap), & = 1,..., n, is a sample from a p-variate distribu- 
tion F with mean zero, where the d and H are as at the beginning of the 
section. As before, let XE = Y = (Y, Y;). Then the rows of Y — E(Y) will 
be independent and have a common distribution, and the n rows of Y, will 
therefore be independently distributed according to d-variate distributions 
F( Jo — 7a: ---» Jad — Naa)» The vectors (3,,..., 5,,); i — 1,..., d, all lie 
in IIg, and under H they are all equal to zero. It follows that if the size of 
the normal-theory test of this reduced problem is robust against nonnormal- 
ity (in F), the test is also robust against nonnormality in the original 
distribution F. In particular, the tests of Examples 1 and 2 are therefore 
robust against nonnormality. 

In some multivariate studies of the kind described in Section 1, observa- 
tions are taken not only on the characteristics of interest but also on certain 
covariates. E 


Example 3. Consider the two-sample problem of Section 3, where 
(X)... X02) and (X$9,..., Xf2) represent p measurements under treatments 1 
and 2 on random samples of n, and n, subjects respectively, but suppose that in 
addition q control measurements (X1), ,,..., X0») and (235,1. s A pta) 
are available on each subject. The n = n, + n, (p + q)-vectors of X's are assumed 
to be independently distributed according to (p + q)-variate normal distributions 
with common covariance matrix and with expectations E(X®) = ¢,, E(X{?) = "i 
for i=1,...,p and E(X®) = E(X) =», for i=p+1,...,p+q. The hy- 
pothesis being tested is H:£, =n; for i=1,..., p. It is hoped that the control 
measurements through: their correlations with the p treatment measurements will 
make it possible to obtain a test with increased power despite the fact that these 
auxiliary observations have no direct bearing on the hypothesis. 


More generally, suppose that the total set of measurements on the ath 
subject is X, — (X,,..., A NIA Xu 544). and that the vectors 
X € — L...,n are independent, ( p + q)-variate normal with common 
covariance matrix. For i — 1,..., p, the mean vectors (£,,..., £,;) are 
assumed as in Section 1 to lie in an s-dimensional subspace II, of n-space, 
the hypothesis specifying that (£,,,...,¢,,) lies in an (s — r)-dimensional 
subspace II, of IIg. For i = p + 1,..., p + q, the vectors (£j... £,) are 
assumed to lie in IL, under both the hypothesis and the alternatives. 
Application of the orthogonal transformation CX of Section 1 to the 
augmented data matrix and some of the invariance considerations of the 
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present section result in the reduced canonical form 


(I A r 
Z V] m=n-s 


Part 


where the r + m rows are independent ( p + q)-variate normal with com- 
mon covariance matrix and means 


E(Y) =» sE) 0 - -E(U) EO E(V) - 0. 


The hypothesis being tested is H : y = 0. This problem bears a close formal 
resemblance to that considered for the model (18), with the important 
difference that the expectations E(U) — v are now assumed to be zero. A 
number of invariant tests making use of the auxiliary variables U and V 
have been proposed, and it is shown in Marden and Perlman (1980) for the 
case r — 1 that some of these are substantially more powerful than the 
corresponding T?-test based on Y and Z alone. For reduction by invari- 
ance, comparative power, and admissibility of various tests in the case of 
general r, see Kariya (1978) and Marden (1983), where there is also a survey 
of the literature. A detailed theoretical treatment of this and related testing 


problems is given by Kariya (1985). 


6. SIMULTANEOUS CONFIDENCE INTERVALS 


the tests and confidence sets of Chapter 7 were 
generalized from the univariate to the multivariate linear model. The present 
section is concerned with the corresponding generalization of Scheffé's 
simultaneous confidence intervals (Chapter 7, Section 9). In the canonical 
form of Section 2, the means of interest are the expectations 1, = E(Y,j), 
i=1,....% Jb RE shall here consider simultaneous confidence 
intervals not for all linear functions EEC; jn; but only those of the form 


In the preceding sections, 


r P P r 
M. uUjni; — y o| X n. 
j=1 j=l ja Miel 
ection 1 in that the same linear 
p components of the multi- 
linear combinations of these 


(1979, 1980).] 


This is in line with the linear hypotheses of S 
function Xu;u;; is considered for each of the 
variate distribution. The objects of interest are lin 
functions. [For a more general discussion, see Wijsman 


als for other linear functions (based on the. Lawley- 


*Simultaneous confidence interv: : 
Hotelling trace test) are discussed by Anderson (1984, Section 8.7.3). 
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When r = 1, one is dealing with a single vector (m,,...,7,), and the 
simultaneous estimation of all linear functions X7. v; is conceptually very 
similar to the univariate case treated in Chapter 7, Section 9. 


Example 4. Contrasts in the s-sample problem. Consider the comparison of two 
products, of which p quality characteristics (£,,,...,,,) and (€2,.--, 2p) are 
measured on two samples. The parametric functions of interest are the linear 
combinations 20,(£;, — £1,). Since for fixed j only the difference £,, — $, is of 
interest, invariance permits restricting attention to the variables Y= (Xj 
X,,)/ V2 and S, and hence r = 1. If instead there are s > 2 groups, one may be 
interested in all contrasts Z; ,w;$,;, Lw, = 0. One may wish to combine the same 
contrasts from the p different components into Ly,(Lw,é,,), Ew, = 0, and is then 
dealing with the more general case in which r = s — 1. 


As in the univariate case, it will be assumed without loss of generality 
that Xu? = 1 so that u € U, and the problem becomes that of determining 
simultaneous confidence intervals 


(20) L(u, 0; y, S) s un» < M(u,v; y, S) forall u& U andallv 


with confidence coefficient y. The argument of the univariate case shows 
that attention may be restricted to L and M satisfying 


(21) L(u,v;y, S) = - M(-u,v; y, S). 


We shall show that there exists a unique set of such intervals that remain 
invariant under a number of groups, and begin by noticing that the problem 
remains invariant under the group G, of Section 2, which leaves the sample 
matrices Y and Z as maximal invariants to which attention may therefore 
be restricted. 

Consider next the group G, of Section 2, that is, the group of orthogonal 
transformations Y* = QY, n* = Qn. The argument of Chapter 7, Section 9 
with respect to the same group shows that L and M depend on u, y only 
through uy and y’y, so that 


L(u,v; y, S) =L,(u'y, y'y; v, S), M(u, v; y, S) = M,(u'y, y'y; v, S). 
Apply next the group Gj of translations Y* = Y + a, n* = + a, where a 


is an arbitrary r X p matrix. Since u'g*o = u'qo + u'av, equivariance Te- 
quires that 


L,(u'(y + a) (y + a) (y + a); v, S) = Ly(u'y, y’y; v, S) + u'a, 
and hence, putting y = 0, L,(0,0; v, S) = L,(v, S), and replacing a by y, 
L,(u'y, y'y; v, S) = u'yo + Ly (v, S) 


and the analogous condition for M. 
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In order to determine L;, consider the group G; of Section 2, that is, the 
group of linear transformations Y* — YB, Z* — ZB, and thus S* — B'SB. 
An argument paralleling that for G, shows that an equivariant L, and M; 
must satisfy 


(22) L,(Bv, S) =L,(v, BSB), | M;(Bv, S) = M,(v, B'SB) 


for all nonsingular B, positive definite S, and all v. In particular, when 
S = I one has 


L,(v,I)- L,(Bv, 1) for all orthogonal B 


so that L,(v, I) = L,(v'v). With B= S~”, so that B'SB = I, and w = 
S~1/v, (22) then reduces to 


L,(w, S) = L;(w‘Sw). 
Thus, 
L(u, v; y, S) = u'yo + L,(v'Sv), M(u, v; y, S) = u'yo + Mj(v'Sv), 


/Sv) = —M,(v’Sv). 
Tr pex K the dés confidence intervals will now be 
completed by an invariance argument that does not involve a transforma- 
tion of the observations (Y, S) but only a reparametrization of the linear 
functions u'qv. If v is replaced by cv for some positive c, then u'qu 
becomes cu'qv, and equivariance therefore requires that 


L,(cv’Svc) = cL,(v'Sv) for all v, $ and c> 0. 


For v'Sv = 1, this gives L(c?) = cLs(1) = ke, say, and hence 
L,(0’Sv) = kV0'Sv.. 


The only confidence intervals satisfying all of the above equivariance 
conditions are therefore given by 


(23) Ju/go — u’yo| < kvv'So forall u€ U andall v. 
constant k, for which the probability (23) equals 


It remains to evaluate the This requires determining the maximum 


the given confidence coefficient y. 
2 
[u(n — »)v] 
max T CES 


(24) uEU, v v'Si 
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For fixed v, it follows from the Schwarz inequality that the numerator of 
(24) is maximized for 


(n-y)v 
yo'(n — »)'(— »)v 


and that the maximum is equal to 


Q5) max [u'(n — y)v]^ = v'(n — »)'(n — »)v. 

ueU 
Substitution of this maximum value into (24) leaves a maximization prob- 
lem which is solved by the following lemma. 


Lemma 2. Let B and S be symmetric p X p matrices, and suppose that S 
is positive definite. Then the maximum of 


v'Bv 
IE 
is equal to the largest root À max of the equation 
(26) |B - AS|- 0, 
and the maximum is attained for any vector v which is proportional to an 


eigenvector corresponding to this root, that is, any v satisfying (B — À max5S)} 
- 0. 


max 


Proof. Since f(cv) = f(v) for all c + 0, assume without loss of gener- 
ality that v'So = 1, and subject to this condition, maximize v'Bv. There 
exists a nonsingular transformation w = Av for which 


vBo-YAwi  vSv-Yw?-1 


where A, zA,2 © > A, are the roots of (26). In terms of the w's it is 
clear that the maximum value of f(v) is obtained by putting w, — 1 and the 
remaining w's equal to zero, and that the maximum value is À}. That the 
maximizing vector is an associated eigenvector is seen in terms of the w's by 
noting that w’ = (1,0,...,0) satisfies (A — A)w = 0, where A is the 
diagonal matrix whose diagonal entries are the J's. 


Application of this lemma, with B = (n — Yy(s— Y), shows that 
[u'(n - Y) 


Jeon v'Sv =A - 9,5), 
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where À} = A,(Y — n, S) is the maximum root of 
(27) KY = n)(¥ - n) -AS| = 0. 


Since the distribution of Y — 7 is independent of 7, the constant k in (23) 
is thus determined by 


P -olà (Y, S) E4 k?] FAY 


and hence coincides with the critical value of Roy's maximum-root test at 
level a = 1 — y. In particular when r = 1, the statistic (m + 1 — p)A,/p 
has the F-distribution with p and m + 1 — p degrees of freedom. 

As in the univariate case, one may wish to permit more general simulta- 
neous confidence sets 


u'nv E A(u, v; y,s) forall ue U, v. 


If the restriction to intervals is dropped, equivariant confidence sets are no 
longer unique, and by essentially repeating the derivation of the intervals it 
is easy to show that (Problem 30) the most general equivariant confidence 
sets are of the form 


(28) w(n- c4 forall wEU andallv, 

v'Sv 
where A is any fixed one-dimensional set. However, as in the univariate 
case, if the confidence coefficient of (28) is y, the set A contains the interval 
(—k, k) for which the probability of (23) is y, and the intervals (23) are 


therefore the smallest confidence sets at the given level. . 
There are three confidence statements which, though less detailed, are 


essentially equivalent to (23): 

(i) It follows from (25) that (23) is equivalent to 
(29) v(-3)(n-»)"3s kAv'Su forall v. 
de simultaneous confidence ellipsoids for all 


the statement 


1 Be ; vectors 
These inequalities provi 


qu. 
(i) Alternatively, 
for all vectors uy, u € 
[u(y yo]? (n 7 urn - y)v 
EA d eo E REM 
v'Sv v'Sv 
th respect to v) of this ratio is the largest 


one may be interested in simultaneous confidence sets 


U. For this purpose, write 


By Lemma 2, the maximum (wi 
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root of 
(30) (n — y)'uu'(n — y) - AS| = 0. 
As was seen in Section 2, with y in place of u’(n — y), this equation has 
only one nonzero root, which is equal to 
u'(n — y)S^ (2 — y)'u, 
and (23) is therefore equivalent to 
(31) w(n-y)S(n—y)usk? forall we U. 


This provides the desired simultaneous confidence ellipsoids for the vectors 
u'n, u € U. 


Both (29) and (31) can be shown to be smallest equivariant confidence 
sets under some of the transformation groups considered earlier in the 
section (Problem 31). 


(iii) Finally, it is seen from the definition of A, that (23) is equivalent to 
the inequalities 


(32) MY = n, S) x kè, 


which constitute the confidence sets for 7 obtained from Roy’s maximum- 
root test. 


As in the univariate case, the simultaneous confidence intervals (23) for 
u'ņv for all u € U and all v have the same form as the uniformly most 
accurate unbiased confidence intervals 


(33) |u'no — u’yo| < koyv'Su 


for a single given u € U and v (Problem 32). Clearly, ky < k, since the 
probability of (33) equals that of (23). The increase from kọ to k is the price 


paid for the stronger assertion, which permits making the confidence 
statements 


là^nó — &'yb| < kVB'SB 


for any linear combinations £76 suggested by the data. 
The simultaneous confidence intervals of the present section were derived 
for the model in canonical form. For Particular applications, Y and S must 


be expressed in terms of the original variables X. (See for example, 
Problems 33, 34.) 
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7. x?-TESTS: SIMPLE HYPOTHESIS AND UNRESTRICTED 
ALTERNATIVES 


UMP invariant tests exist only for rather restricted classes of problems, 
among which linear hypotheses are perhaps the most important. However, 
when the number of observations is large, there frequently exist tests which 
possess this property at least approximately. Although a detailed treatment 
of large-sample theory is outside the scope of this book, we shall indicate 
briefly some theory of two types of tests possessing such properties: x?-tests 
and likelihood-ratio tests. In both cases the approximate optimum property 
is a consequence of the asymptotic equivalence of the problem with one of 
testing a linear hypothesis. This relationship will be sketched in the next 
section. As preparation we discuss first a special class of x? problems. 

It will be convenient to begin by considering the linear hypothesis model 
with known covariance matrix. Let Y = (Y,, ..., ¥,) have the multivariate 
normal probability density 


(34) |A] 1 
1,93p| 7 5 
(27)! 2 


q 4 
2m a; i - v) - 7) 


i=l j=1 


with known covariance matrix A~}. The point of means 7 = (1... 7,) is 
known to lie in a given s-dimensional linear space Ig with s sq the 
hypothesis to be tested is that 7 lies in a given (s — r)-dimensional linear 
subspace IL, of Hg (r <s). This problem (which was considered in 
canonical form in Section 4) is invariant under a suitable group G of linear 
transformations, and there exists a UMP invariant test with respect to G, 


given by the rejection region 
(35) YXaj( F a)(y zi iy) 5 YXaj0i 3 addy F îy) 
= EZ a;l- &)(à, - y) 
E 
which is closest to the sample point y in the 


that is, which minimizes 
arly 7) is the point in 


Here ij is the point of Ig wh 
metric defined by the quadratic form rae pid n T 
Ly quantity Lz; (yi a nO; Tm n) for yin a- Simi 
, Minimizing this quantity. ; ?-distribu- 
When theiks a is true, the left-hand side of (35) has a x7-distribu 
tion with r degrees of freedom, so that C is determined by 


(36) [59 d; = a. 
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When 7 is not in II,, the probability of rejection is 
co 

(37) J paz) ae, 
(s 


where p,(z) is the noncentral x?-density of Chapter 7, Problem 2 with r 
degrees of freedom and noncentrality parameter M, obtained by replacing 
Y; ip 9, in (35) with their expectations, or equivalently, if (35) is considered 
as a function of y, by replacing y with ņ throughout. This expression for 
the power is valid even when the assumed model is not correct so that 
E(Y) ^ » does not lie in Ig. For the particular case that n € IIg, the 
second term in this expression for X^ equals 0. A proof of the above 
statements is obtained by reducing the problem to a linear hypothesis 
through a suitable linear transformation. (See Problem 35). 

Returning to the theory of x?-tests, which deals with hypotheses concern- 
ing multinomial distributions," consider n multinomial trials with m possi- 
ble outcomes. If p —(p,,..., Pm) denotes the probabilities of these 
outcomes and X, the number of trials resulting in the ith outcome, the 
distribution of X = (X,,..., Xm) is 


n! 


(38) P(x,,...,x,) = pmo (Xx-2^» XEn-1) 


— p 
xj..cx, 1 P1 


The simplest x? problems are those of testing a hypothesis H : p = 7 where 
T =(m,..., Tm) is given, against the unrestricted alternatives p # 7. AS 
n > co, the power of the tests to be considered will tend to one against any 
fixed alternative. (A sequence of tests with this property is called consistent.) 
In order to study the power function of such tests for large n, it is of interest 
to consider a sequence of alternatives p) tending to 7 as n — oo. If the 
rate of convergence is faster than 1/ yn, the power of even the most 
powerful test will tend to the level of significance o. The sequences reflecting 
the aspects of the power that are of greatest interest, and which are most 
likely to provide a useful approximation to the actual power for large but 


finite n, are the sequences for which yn ( p — m) tends to a nonzero limit, 
so that 


^; 
(39) p = se e+ RP 
n 


say, where yn R(? tends to zero as n tends to infinity. 


*For an alternative approach to such hypotheses see Hoeffding (1965). 
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Let 


(40) y- MaM 
C 
Then X7 ,Y, = 0, and the mean of Y, is zero under H and tends to A, under 


the alternatives (39). The covariance matrix of the Y 's is 
(41) 9,; = -TT if i5 j, o, = 7(1— 7) 


when H is true, and tends to these values for the alternatives (39). As 
n co, the distribution of Y = (Y,,..., Y, .;) tends to the multivariate 
normal distribution with means E(Y,) = 0 under H and E(Y,) = A, for the 
sequence of alternatives (39), and with covariance matrix (41) in both cases. 
[A proof assuming H is given for example by Cramér (1946, Section 30.1). 
It carries over with only the obvious changes to the case that H is not true.] 
The density of the limiting distribution is 


m-1 a 
Leg n) 
(42) cexp af ye mou 2 


i=l i 


and the hypothesis to be tested becomes HUM E Am-1 7 0. ad 
According to (35), the UMP invariant test in this asymptotic m le 


rejects when 


and hence when 


(43) ay eae 


36) with r=m—1. [The 


i ined b 
where ; oe RE e y C distribution of the test 


accuracy of the x2-approximation to the exact 
dentine M this i“ is discussed for example by Radlow and n c Tia 
more accurate approximations in this and related problems, se 


(1985) and the literature cited there.] The limiting power of the test against 
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the sequence of alternatives (39) is given by (37) with X = X? ,A2/7;. This 
provides an approximation to the power for fixed n and a particular 
alternative p if one identifies p with p‘™ for this value of n. From (39) one 
finds approximately A, = Vn ( p, — v,), so that the noncentrality parameter 
becomes 


E LAE 


Tj 


(44) A m 


i=l 


Example 5. Suppose the hypothesis is to be tested that certain events (births, 
deaths, accidents) occur uniformly over a stated time interval such as a day or a 
year. If the time interval is divided into m equal parts and p, denotes the 
probability of an occurrence in the ith subinterval, the hypothesis becomes H : p, = 
1/m for i = 1,..., m. The test statistic is then 


mE eb 


i=l 


where v, is the relative frequency of occurrence in the ith subinterval. The 
approximate power of the test is given by (37) with r=m—1 and Y= 
mni lp; — (7m. 


Unbiasedness of the test (43) and a local optimality property among tests 
based on the frequencies v, are established by Cohen and Sackrowitz (1975). 

Example 5 illustrates the use of the x?-test (43) for providing a particu- 
larly simple alternative to goodness-of-fit tests such as that of Kolmogorov, 
mentioned at the end of Chapter 6, Section 13. However, when not only the 
frequencies v, but also the original observations X, are available, reduction 
of the data through grouping results in tests that tend to be less efficient 
than those based on the Kolmogorov or related statistics. For further 
discussion of x? and its many generalizations, comparison with other 
goodness-of-fit tests, and references to the extensive literature, see Kendall 
and Stuart (1979, Section 30.60). The choice of the number m of groups is 
considered, among others, by Quine and Robinson (1985) and by 
Kallenberg, Oosterhoff, and Schriever (1985). 


8. x'-AND LIKELIHOOD-RATIO TESTS 


It is both a strength and a weakness of the x?-test of the preceding section 
that its asymptotic power depends only on the weighted sum of squared 
deviations (44), not on the signs of these deviations and their distribution 
over the different values of i. This is an advantage if no knowledge is 
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available concerning the alternatives, since the test then provides equal 
protection against all alternatives that are equally distant from H : p = 7 in 
the metric (44). However, frequently one does know the type of deviations 
to be expected if the hypothesis is not true, and in such cases the test can be 
modified so as to increase its asymptotic power against the alternatives of 
interest by concentrating it on these alternatives. 

To derive the modified test, suppose that a restricted class of alternatives 
to H has been defined 


K:peS,p*rm. 
Let the surface 5^ have a parametric representation 
p, Efí05....0), USI amy 
and let 
m= f (00,..., 62). 


Suppose that the 6, are real-valued, that the derivatives 0/,/00, exist and 
are continuous at 6°, and that the Jacobian matrix (0f,/00,) has rank s at 
6°. 1f 8? is any sequence such that 


(45) Ws (0j - 6°) 8, 


the limiting distribution of the variables (Y... 
section is normal with mean 


, Y, 4) of the preceding 


s Of, 
)-A-Y5— 
(46) E(Y) - A, LEY» : 


i i is i ing f, about the point 0° 
and covariance matrix (41). This is seen by expanding f; à : 

and applying the limiting distribution (42). The problem of MH 
against all sequences of alternatives in K satisfying (45) is ther: 
asymptotically equivalent to testing the hypothesis 


Amo =An-1 70 


s | $ ves K (A... AS) € Hg where 
in the family (42) against the alternatives K : (Ay --» Àm : 
Ils, is denied dr formed by the totality of points with coordinates 


s Of; 
(47) A= LN 
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We note for later use that for any fixed 7, the totality of points 


A; 
pit "D i-]1,...,m, 
n 


with the A, satisfying (47), constitute the tangent plane to 5^ at 7, which 
will be denoted by F. 

Let (A,,..., Âm) be the values minimizing £7 ,(y; — A,)?/7, subject to 
the conditions (A,,..., A,,_,) € Ig and A,, = —(A, + --- *A,, ,). Then 
by (35), the asymptotically UMP invariant test rejects H in favor of K if 


Xx EG-ày Ew 
i=l deest acd diui > GC. 
T; TMi Ti 


or equivalently if 


m 2 m m 
n Y (vi-m) nd (v,- 5) nE (P-n) 
(48) i=l = i=l & i=l > (ek 
Tj Tj Nj 

where the 5; minimize Y(v, — p;)?/7, subject to p € 7. The constant C is 
determined by (36) with r = s. An asymptotically equivalent test, which, 
however, frequently is more difficult to compute explicitly, is obtained by 
letting the ; be the minimizing values subject to p € ^ instead of p € I. 
An approximate expression for the power of the test against an alternative p 
is given by (37) with X obtained from (48) by substituting p; for v; when 
the f; are considered as functions of the »;. 


, Example 6. Suppose that in Example 5, where the hypothesis of a uniform 
distribution is being tested, the alternatives of interest are those of a cyclic 
movement, which may be represented at least approximately by a sine wave 


1 i2m/m 
P= kp sin( u — 0) du, px e. nt. 
m (i-)20/m ( e d 


Here p is the amplitude and @ the phase of the cyclic disturbance. Putting 
£ = pcos 9, n = psin9, we get 


1 
p= (+a + bn), 
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where 
LM 7 7 7 
a; = 2msin— sin(2i - 1) —, — b, = —2msin— cos(2i - 1)—. 
m m m m 


The equations for p; define the surface 5^, which in the present case is a plane, so 
that it coincides with 2. 
The quantities $, $ minimizing Y(v, — p;)^/z, subject to p € ^ are 


with 7, = 1/m. Let m > 2. Using the fact that Ya, = Eb, = La,b, = 0 and that 


Y ar Qi 


m. T m 
?(2i-1— =>, 
= 2 cos? (2i js: 7 


i=l 


the test becomes after some simplification 
m PR m PAL 
zl bunt - Dž] + PEE mate 
jm 


where the number of degrees of freedom of the left-hand side is s = 2. The 
noncentrality parameter determining the approximate power 1s 


T7y 7y 4 tg ed 
2 put sin—| = np m sin —. 
X n( ém sin) + n( am z) p = 


The x?-tests discussed so far were for simple hypotheses. Consider now 
the more general problem of testing H: p € against the alternatives 
K: pe, pT where TCS and where 5^ and 7 have parametric 


representations 
0 
PS: p, o f (8). 8); T: p = (8009 baes b): 


i i i blem is the fact that for 
The basis for a large-sample analysis of this pro i 

large n a sphere of od p/ yn can be located which for "yv Ne p 
contains the true point p with arbitrarily high proboha ‘A m jmd 
therefore be restricted to sequences of points p € 4$ whic 


fixed point meJ at the rate of 1/ hn. Oe Ra "d ic) 
f (00, ..., 89), and let 0 be a sequence satisfying (45). iance matrix 


(Y,,..., Y, ,) have a normal limiting distribution xm Eu bloc id 
(41) and a vector of means given by (46). Let IIa backend : 
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II, be the linear space 


s op, 
MEAS DA|, | 
jr ` 06, 9? 


and consider the problem of testing that p(" is a sequence in H for which 
0 satisfies (45) against all sequences in K satisfying this condition. This is 
asymptotically equivalent to the problem, discussed at the beginning of 
Section 7, of testing (A,,...,A,,_,) € IL, in the family (42) when it is 
given that (A,,..., 4,, ,) € IIg. By (35), the rejection region for this 
problem is 


where the A, and A, minimize L(y, — A,)?/z, subject to A, = 
—(A, + +--+ tA, 4) and (A,,..., A, ,) in Ig and II, respectively. In 
terms of the original variables, the rejection region becomes 


(49) es >C. 


T; 


n} (v; —h) 74 n} (r, > pi) 


Here the f, and f, minimize 


(»,— Pi) 


T, 


(50) y» 


when p is restricted to lie in the tangent plane at z to 5^ and 7 
respectively, and the constant C is determined by (36). 

The above solution of the problem depends on the point 7, which is not 
given. A test which is asymptotically equivalent to (49) and does not depend 
on 7 is obtained if p, and p, are replaced by p* and p** which minimize 
(50) for p restricted to 5^ and J instead of to their tangents, and if further 
7, is replaced in (49) and (50) by a suitable estimate, for example by »;. This 
leads to the rejection region 


(51) ny Gp -aX (», = pi)’ Lay (pf -py s'ci 


"i Vi V; 
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where the p** and př minimize 


pem 


7i 


(52) 


subject to p € FJ and p € F respectively, and where C is determined by 
(36) as before. An approximation to the power of the test for fixed n and a 
particular alternative p is given by (37) with X obtained from (51) by 
substituting p; for v, when the p* and pf* are considered as functions of 
the »,.* 

A more general large-sample approach, which unlike x? is not tied to the 
multinomial distribution, is based on the method of maximum likelihood. 
We shall here indicate this theory only briefly, and in particular shall state 
the main facts without the rather complex regularity assumptions required 
for their validity.* 

Let p(x), 6 = (8,,..., 0,), be a family of univariate probability densi- 
ties, and consider the problem of testing, on the basis of a (large) sample 
X,..., X, the simple hypothesis H:6,— 09, i- 1,...,r. Let 
(ê ai RR 6) be the maximum-likelihood estimate of 6, that is, the parameter 
vector maximizing pa(xı)--- Po(X,)- Then asymptotically as n > gd 
tion can be restricted to the Ê, since they are “asymptotically sufficient”. 
The power of the tests to be considered will tend to one against any fixed 
alternative, and the alternatives of interest, as in the x? case, are sequences 


6'") satisfying 
(53) yn (8 sS 0°) a [Ne 


If Y, = /n (6, — 69), the limiting distribution of Y, ..., Y, is the multi- 
variate normal distribution (34) with 


log p(X) 
d a= e (09) = jer 


6-0 


and with n, = 0 under H and n; = A, for the alternatives satisfying (53). 
i 
: i icall 

1A proof of the above statements and a discussion RES codec pine rie, 
equivalent to (48) and sometimes easier to determine explicitly are given, 
Hodges, and Lehmann (1959). . s 

For a detailed treatment and references to the literature 9e Bip ufficiency and further 

5This was shown by Wald (1943); for à definition of asymptotic $ 
Tesults concerning this concept see LeCam (1956, 1960). 


ing (1980, Section 4.4). 
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By (35), the UMP invariant test in this asymptotic model rejects when 


r f: 
(55) = E Y an(ô,— 0°)(4, — 8°) > C. 
i=l j=l 
Under H, the left-hand side has a limiting x?-distribution with r degrees of 
freedom, while under the alternatives (53) the limiting distribution is non- 
central x? with noncentrality parameter 


r r 
(56) X = lim x Y a(o” - ep)(at — ep). 
i=] j= 
The approximate power against a specific alternative @ is therefore given by 
(37), with A? obtained from (56) by substituting @ for 0. 
The test (55) is asymptotically equivalent to the likelihood-ratio test, 
which rejects when 


(57) de string oap (ngos 


This is seen by expanding Y". log pyo(x,) about Y7..,log py(x,) and using 
the fact that at @ = Ê the derivatives dLlog p,(x,)/906, are zero. Applica- 
tion of the law of large numbers shows that —2log A, differs from the 
left-hand side of (55) by a term tending to zero in probability as n — oo. In 
particular, the two statistics therefore have the same limiting distribution. 

The extension of this method to composite hypotheses is quite analogous 
to the corresponding extension in the x? case. Let 0 = (0,,.... 9) and 
H:60,— 09 for i=1,...,r (r < s). If attention is restricted to sequences 
9°” satisfying (53) for i — 1,...,s and some arbitrary 65, ,,..., 0), the 
asymptotic problem becomes that of testing 3; = =- =17,=0 against 
unrestricted alternatives (,,..., 7,) for the distributions (34) with a;j = 
a,;(8°) given by (54). Then $j, = Y, for all i, while 5, = 0 for i = 1... 7 
and = Y, for i =r + 1,...,5, so that the UMP invariant test is given by 
(55). The coefficients a; = a;,(0°) depend on 6°, ,,..., 6? but as before an 
asymptotically equivalent test statistic is obtained by replacing a;, (0^) with 
a, (8). Again, the statistic is also asymptotically equivalent to minus twice 
the logarithm of the likelihood ratio, and the test is therefore asymptotically 
equivalent to the likelihood-ratio test,* which rejects when 


— piGa) +++ (xn) 


69) NW CAES CA, 


*The asymptotic theory of likelihood-ratio tests has been extended to more general types of 
problems, including in particular the case of restricted classes of alternatives, by Chernoff 
(1954). See also Serfling (1980). 
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where 6 is the maximum-likelihood estimate of 6 under H, and where 
—2log A, as before has a limiting x?-distribution with r degrees of 
freedom. 


Example 7. Independence in a two-dimensional contingency table. In generaliza- 
tion of the multinomial model for a 2 X 2 table discussed in Chapter 4, Section 6, 
consider a twofold classification of m subjects, drawn at random from a large 
population, into classes A,,...,A, and B;,..., B, respectively. If n;, denotes the 
number of subjects belonging to both A; and B,, the joint probability of the ab 
variables n,, is 


(59) Tele (En;-», En, 71). 


i,j 


The hypothesis to be tested is that the two classifications are independent, that is, 
that p,, is of the form 


(60) H: pij = Pi; 


for some p,, p’ satisfying X p, = Lp; = 1. j "- 

hieman EA penis tests are provided by (51) and the likeli- 
hood-ratio test. Since the minimization required by the former leads to a system of 
equations that cannot be solved explicitly, let us consider the likelihood-ratio 
approach. In the unrestricted multinomial model, the probability (59) is maximized 


by p,, = n,,/n; under H, the maximizing probabilities are given by 


where n,.=¥n,,/b and n.j = X;nj/a (Problem 39). Substitution in (58) gives 


Ju 
TI 
cate 
Tne nS 
i J 


Since under @ the p,, are subject only to the restriction ey ot Ww mw 
$= ab — 1. Similarly, s 77 (2-0 * (b — D s pays (a= D - D 
has a limiting x^-distribution with 7 = (ab sie and possible improve- 
degrees of freedom. The accuracy of the X^apprOXime, And Upton (1984) and 
ments, in this and related problems are diste cited in these papers. 
Lewis, Saunders, and Westcott (1984), aaa oil contingency tables, see for 
For further work on two- and higher- ; and Holland (1975), and 
example the books by Haberman (1974), vx iia 
Plackett (1981), and the paper by Goodman (1783). 
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9. PROBLEMS 
Section 2 
(i) If m < p, the matrix S, and hence the matrix $/m (which is an unbiased 
estimate of the unknown covariance matrix of the underlying p-variate 
distribution), is singular. If m > p, it is nonsingular with probability 1. 


(i) If r+m<p, the test $(y, u, z) =a is the only test that is invariant 
under the groups G, and G, of Section 2. 


[(ii): The U’s are eliminated through G,. Since the r + m row vectors of the 
matrices Y and Z may be assumed to be linearly independent, any such set of 
vectors can be transformed into any other through an element of G;.] 


@ If p«r m, and V= Y'Y, S= Z'Z, the p X p matrix V+S is 
nonsingular with probability 1, and the characteristic roots of the equa- 
tion 


(61) |V-\(V+s)| =0 


constitute a maximal set of invariants under G,, G, and G;. 

(ii) Of the roots of (61), p — min(r, p) are zero and p — min (m, p) are 
equal to one. There are no other constant roots, so that the number of 
variable roots, which constitute a maximal invariant set, is min (7, p) + 
min (m, p) — p. 


[The multiplicity of the root À — 1 is p minus the rank of S, and hence 
p — min(m, p). Equation (61) cannot hold for any constant A 0,1 for 
almost all V, S, since for any p * 0, V + pS is nonsingular with probability 1.] 


G) IfA and B are k X m and m X k matrices respectively, then the product 
matrices 4B and BA have the same nonzero characteristic roots. 


(i) This provides an alternative derivation of the fact that W defined by (6) is 
the only nonzero characteristic root of the determinantal equation (5). 


[G): If x is a nonzero solution of the equation ABx = Ax with À 0, then 
y = Bx is a nonzero solution of BAy = A y.] 


In the case r — 1, the statistic W given by (6) is maximal invariant under the 
group induced by G, and G, on the statistics Y, U, (i=1,...,p; “= 
1,...,s — 1), and S = ZZ. 

[There exists a nonsingular matrix B such that B’SB = J and such that only 
the first coordinate of YB is nonzero. This is seen by first finding B, such that 
Bi SB, = I and then an orthogonal Q such that only the first coordinate of 
YB,Q is nonzero.] 


Let Z,, (a = 1,...,m; i=1,..., p) be independently distributed as N(0, 1), 
and let Q — Q(Y) be an orthogonal m X m matrix depending on a random 
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variable Y that is independent of the Z's. If Z* is defined by 


(21... 2$) = (Zu. Zt) Q's 


then the Z% are independently distributed as N(0,1) and are independent 
of Y. 

[For each y, the conditional distribution of the (Zj,... Z,,,) Q'( y), given 
Y — y, is as stated.] 


Let Z be the m X p matrix (Z,,), where p < m and the Z,, are independently 
distributed as N(0,1), let $ = Z’Z, and let S, be the matrix obtained by 
omitting the last row and column of S. Then the ratio of determinants |S|/|S,| 
has a x?-distribution with m — p + 1 degrees of freedom. 

[Let q be an orthogonal matrix (dependent on Zj;,..., Zm) such that 


(Zu... Za)Q! 2 (R 0 ... 0), where R? = Et Za: Then 
FINE ME s pucr omma 
Zh Zh ve Zm||o Zh dp 
S = Z'Q'QZ = EUAN A ree " 
ze zem I MEE Rr EZEN 


where the Z* denote the transforms under Q. The first of the matrices on the 
right-hand side is equal to the product 


latila) 


where Z* is the (m — 1) X (p — 1) matrix with elements Zh (a= 2.4 5 
i=2,...,p), T à the (p — 1) X (p — 1) identity matrix, Zr is cma 
vector (Z5 ... Zj,)’, and 0 indicates a row or column of zeros. It follows - 
|S| is equal to R? multiplied by the determinant of Z*Z*. e" Si y " 
product of the m X (p — 1) matrix obtained by omitting the last enm fe 
multiplied on the left by the transpose of this m x ( p- 1) apy l i beh ag 
to R? multiplied by the determinant of the matrix obtained by omi adr 
last row and column of Z*’Z*. The ratio |S|/]S;] has therefore hen am A 
to the corresponding ratio in terms of the Z7, with m and p bie i y 
m — 1 and p — 1, and by induction the problem is seen to be wac RSI 
and p are replaced by m — k and p — k for any k « p. In parti a tfe 
can be evaluated under the assumption that m and p Lib dum ra d 
m-(p-1)andp-(p-D7l In this case, the mats P which v 
(Zu... Z, 1i); the determinant of S is |S|= Eazi es ir 
XŽ, -p , distribution; and since S is a 1 X 1 matrix, |S;| is rep y 1. 


istic W = YS-!Y' defined by (6), 
Null distribution of Hotelling's T^. The statistic W YS s bos d S 9 
where Y is a row vector, has the distribution of a ratio, of wl 
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and denominator are distributed independently, as noncentral x? with non- 
centrality parameter i? and p degrees of freedom and as central x^ with 
m + 1 — p degrees of freedom respectively. 

[Since the distribution of W is unchanged if the same nonsingular transforma- 
tion is applied to (Y,,..., ¥,) and each of the m vectors (Z,,,-..,Z,,), the 
common covariance matrix of these vectors can be assumed to be the identity 
matrix. Let Q be an orthogonal matrix (depending on the Y's) such that 
(X, ... ¥%)Q@=(0 ... T), where T? = XY. Since QQ’ is the identity 
matrix, one has 


W = (YQ)(Q's-o)(o'v) = (o --- OT)/(Q'S-19)(0 ++ 0 ry. 


Hence W is the product of T?, which has a noncentral x?-distribution with p 
degrees of freedom and noncentrality parameter 4°, and the element which 
lies in the pth row and the pth column of the matrix Q’S~'Q = (Q'SQ) ! = 
(Q'Z’ZQ)_*. By Problems 5 and 6, this matrix is distributed independently of 
the Y's, and the reciprocal of the element in question is distributed as 
XR cr] 

Note. An alternative derivation of this distribution begins by obtaining the 
distribution of S, known as the Wishart distribution. This is essentially a 
p-variate analogue of x? and plays a central role in tests concerning covari- 
ance matrices. [See for example Seber (1984).] 


Section 3 


Let (X,,..., Xap), a = 1,..., n, be a sample from any p-variate distribution 
with zero mean and finite nonsingular covariance matrix X. Then the distri- 
bution of T? defined by (10) tends to x? with p degrees of freedom. 


The confidence ellipsoids (11) for (£,,..., £,) are equivariant under the groups 
G,-G, of Section 2. 


The two-sample test based on (12) is robust against nonnormality as n, and 
n > oo. 


The two-sample test based on (12) is robust against heterogeneity of covari- 
ances as nj and n, > oo when n/n, > 1, but not in general. 


Inversion of the two-sample test based on (12) leads to confidence ellipsoids 
in Ms vector (pe Uim f — £07) which are uniformly most accurate 
equivariant under the groups G,-G, of Section 2. 


Simple multivariate regression. In the model of Section 1 with 
(62) $72 + Bt, (v1: .,2; i= 1,5), 
the UMP invariant test of H: B, = --- = f, = 0 is given by (6) and (9), with 


y= Ê, S; zi X [X FIG, = Bit.) [Xj I à; n Bj] 


where B, = EX, (t, — )/ E(t, — i), & = X, — B, 
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ian Let-(Y heses Y) v= l,...,n, be a sample from a p-variate distribution F 
with mean zero and covariance matrix X, and let Z/ = Y c, Y,/ /E7.,c2 
for some sequence of constants c;, c;,... . Then (Z^... , Z/) tends in law 
to N(0, Z) provided the c's satisfy the condition (50) of Chapter 7. 

[By the Cramér-Wold theorem [see for example Serfling (1980)], it is enough to 
prove that Za, Z™ + N(0, a/Za) for all a = (a;,...,a,) with Ea? = 1, and 
this follows from Lemma 3 of Chapter 7.] 

15. Suppose X, =$, + Upp where the £,, are given by (62) and where 
(Unas D Joe m l,...,n, is a sample from a p-variate distribution with 
mean 0 and covariance matrix X. The size of the test of Problem 13 is robust 


for this model as n > oo. 
[Apply Problem 14 and the univariate robustness result of Chapter 7, Section 


8] 
Note. This problem illustrates how the robustness of a univariate linear test 
carries over to its multivariate analogue. For a general result see Arnold (1981, 
Section 19.8). 
Section 4 

16. Verify the elements of V and S given by (14) and (15). 

17. Let V and S be p X p matrices, V of rank a < p and S nonsingular, and let 
A,,..., À, denote the nonzero roots of |V — AS| = 0. Then 


G) p; 71/0 +A), i 7L... a, are the a smallest roots of 
(63) [S—p(V + S)|=0 


(the other p — a being = 1); 
Gi) v, =1 + À; are the a largest roots of 


(64) IV S 05|=0. 


18. Under the assumptions of Problem 17, show that 


1 WI 
II 14A, W+Sl 
of a matrix is equal to the product of its characteristic roots.) 


one nonzero root, then B is of rank 1. In canonical form 
then exists a vector (aj. 4p) and constants 


[The determinant 
19. (i) If (13) has only 


= 1,.. 7. 

(65) (Mea rees9 Mop) = ECs» p) for v 
Gi) For the s-sample problem considered in Section 4, restate (65 
the means (£(^^.--» £() of the text. 


) in terms of 
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Let (X4,..., Xap) a= 1,..., n, be independently distributed according to 
p-variate distributions F(x,, — £4,..., Xap — §ap) with finite covariance ma- 
trix E, and suppose the £'s satisfy the linear model assumptions of Section 1. 
Then under H, S,,/(n — s) tends in probability to the (ij)th element aj; 
of X. 

[See the corresponding univariate result of Chapter 7, Section 3.] 

Let (XXP,..., XL); a — 1,...,n,, k=1,...,8, be samples from p-variate 
distributions F(x, — &{",..., x, — §(*)) with finite covariance matrix 2, and 
let A,,..., À, be the nonzero roots of (16) and (A¥,..., A*,) those of (17), with 
V and S given by (14) and (15). Then the joint distribution of ((n — 
5)À,,..., (n — s)À,) tends to that of (Af,..., M5) as n > oo. 


Give explicit expressions for the elements of V and S in the multivariate 
analogues of the following situations: 


(i) The hypothesis (34) in the two-way layout (32) of Chapter 7. 
(i) The hypothesis (34) in the two-way layout of Section 6 of Chapter 7. 


(ii) The hypothesis H' : y;; = 0 for all i, j, in the two-way layout of Section 
6 of Chapter 7. 


The probability of a type-I error for each of the tests of the preceding problem 
is robust against nonnormality: in case (i) as b > oo; in case (ii) as mb — 00; 
in case (iii) as m > oo. 


Section 5 
The assumptions of Theorem 6 of Chapter 6 are satisfied for the group (19) 
applied to the hypothesis H : ņ = 0 of Section 5. 


Let X,;, G7 1,...,a; j 7 1..., b), v — 1,..., n, be n independent vectors, 
each having an ab-variate normal distribution with covariance matrix X and 
with means given by 


E(X,)) ^n * a; * Bj, Xa, = XB, - 0. 


(i) For testing the hypothesis H: œ = --- = a, = 0, give explicit expres- 
sions for the matrices Y and Z of (18) and the parameters ņ = E(Y) 
being tested. 


(ii) Give an example of a situation for which the model of (i) might be 
appropriate. 


Gene both parts of the preceding problem to the two-group case in which 
XN), (A= 1,...,m) and Xf) (» 9 1,...,n3) are n, + n; independent vec- 
tors, each having an ab-variate normal distribution with covariance matrix È 
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27. 


28. 


29. 
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and with means given by 


D ss 
E(X) =m eae BP. E) = pa + af gs 


Ya = Ya - 0, ra = Y^ =0, 
and where the hypothesis being tested is 


H: of) = --- =a) =a) = +. =a? =0. 


As a different generalization, let (Xy... , Xhup) be independent vectors, each 
having a p-variate normal distribution with common covariance matrix 2 and 


with expectation 


El Xyu) = HO tof + B^, Xa - YR? -0  foralli, 
a » 


and consider the hypothesis that each of p®, af, 8O (À= 1... a; » 7 

1,..., b) is independent of i. 

(i) Give explicit expressions for the matrices Y and Z and the parameters 
n = E(Y) being tested. 

(i) Give an example of a situation in which this problem might arise. 


the model assumptions made at the 


Let X be an n X p data matrix satisfying 
* = CX, where C is an orthogonal 


beginning of Sections 1 and 5, and let X 
matrix, the first s rows of which span Ilg. If Y* and Z denote respectively 


the first s and last n — s rows of X*, then E(Y*) = q" say, and E(Z) = 0. 
Consider the hypothesis Hy : U’n*V = 0, where U’ and V are constant matrices 
of dimension a X s and p X b and of ranks a and b respectively. 


(i) The hypotheses of both Section 1 and Section 5 are special cases of Ho. 
(ii) The problem can be put into canonical form Y** (s Xp) and Z** 
((n — s) X p), where the n rows of Y** and Z** are independent 
p-variate normal wi matrix and with means 
E(Y**)= n**, and where Ho becomes Ho: 17" = 0 for all i= 1,...,4, 
j-7l..b. 

Gii) Determine groups leaving this problem invariant and for which the first 
a columns of Y** are maximal invariants, so that the problem reduces 


to a multivariate linear hypothesis in canonical form. 


Consider the special case of the preceding problem in which a = b=1, and let 
U’ eum (ues Ma) US (vis: - Up). Then for testing Hy: uto = 0 


there exists a UMP invariant test which rejects when u'y*o/(v'Sv)u'u > c. 
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Section 6 


The only simultaneous confidence sets for all u’nv, u € U, v that are 
equivariant under the groups G;-G; of the text are those given by (28). 


Prove that each of the sets of simultaneous confidence intervals (29) and (31) is 
smallest among all families that are equivariant under a suitable group of 
transformations. 


Under the assumptions made at the beginning of Section 6, show that the 
confidence intervals (33) 


(i) are uniformly most accurate unbiased, 
(ii) are uniformly most accurate equivariant, and 
(iii) determine the constant ko. 


Write the simultaneous confidence sets (23) as explicitly as possible for the 
following cases: 


(i) The one-sample problem of Section 3 with n; = ; (i = 1,..-, p). 

(ii) The two-sample problem of Section 3 with n; = £(? — £f". 

Consider the s-sample situation in which (X(P,..., XP), v= L-. "te 
k — 1,...,5, are independent normal p-vectors with common covariance ma- 
trix X and with means (£(^,..., £?). Obtain as explicitly as possible the 
smallest simultaneous confidence sets for the set of all contrast vectors 
Cuti, ... Eu, £9), Eu, = 0. 

[Example 10 of Chapter 7 and Problem 16.] 


Section 7 
The problem of testing the hypothesis H: y € IL, against 7 € IIo... when 
the distribution of Y is given by (34), remains invariant under a suitable group 
of linear transformations, and with respect to this group the test (35) is UMP 
invariant. The power of this test is given by (37) for all points (m... 14) 


Let X,,..., X, be iid. with cumulative distribution function F, let a; < ''' 

< a,,_, be any given real numbers, and let ay = —00, a,, = co. If v; is the 

number of X's in (a; ,, a;), the x?-test (43) can be used to test H : F=f 

with 7, = F,(a;) — Fy(a;_,) for i=1,..., m. 

(i) Unlike the Kolmogorov test, this x?-test is not consistent against all 
F, + Ñ as n > oo with the a’s remaining fixed. 

(i) The test is consistent against any F, for which 


F(a)) - F(a) + (aj) - (2,1) 


for at least one i. 
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38. 


39. 
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Section 8 
Let the equation of the tangent J^ at 7 be p, = 7,(1 + ang, + «+: +a,¢,), 


and suppose that the vectors (a,,,...,a,,) are orthogonal in the Ae 
La, aj, = 0 for all k #1. * Bi sense thai 


© 1d »... ) minimizes L(y, — p,)?/z, subject to p EF, th + 
Da AEGA Pi) /' jj o p en 7 


ij”i 


(i) The test statistic (48) for testing H: p = 7 reduces to 


j=l \i=l 


n T | X am) 


Y aym 


i=l 


In the multinomial model (38), the maximum-likelihood estimators p, of the 
p's are f, = x,/n. 

[The following are two methods for proving this result: (i) Maximize 
log P(x,,..., Xm) subject to Zp; = 1 by the method of undetermined multi- 
pliers. (ii) Show that [Tp < II(x,/n)* by considering n numbers of which x, 
are equal to p,/x, for i=1,...,m and noting that their geometric mean is 
less than or equal to their arithmetic mean.] 


In Example 7, show that the maximum-likelihood estimators Pj, Îi, and pj 
are as stated. 


In the situation of Example 7, consider the following model in which the row 
margins are fixed and which therefore generalizes model (iii) of Chapter 4, 
Section 7. A sample of n, subjects is obtained from class A, (i = 1... 4), the 
samples from different classes being independent. If nj, is the number of 
subjects from the ith sample belonging to Bi lh. b), the joint distribu- 
tion of (n5,..., nip) iS multinomial, say, M(";; Py: Py Determine the 
likelihood-ratio statistic for testing the hypothesis of homogeneity that the 
vector ( Pijs- -> Poi) is independent of i, and specify its asymptotic distribu- 


tion. 


The hypothesis of symmetry in a square two" 
one of the responses 4i,..-; 
Occasions (e.g. before and 
subjects whose responses on 
tion of the n;; is arn o 

= p; i,j, y i p 
eS i» likelihood-ratio statistic for testing H, and 


specify its asymptotic distribution. [Bowker (1948).] 
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In the situation of the preceding problem, consider the hypothesis of marginal 
homogeneity H' : p, = p+; for all i, where pj, = EP aPigsD« 7 Ej- Pi: 


() The maximum-likelihood estimates of the p,, under H’ are given by 
Di = ni/Q + À; — X), where the A’s are the solutions of the equations 
Ejn/ü +A, —A,) = Ejn;/ +A, — À). (These equations have no 
explicit solutions.) 

Gi) Determine the number of degrees of freedom of the limiting x?-distribu- 
tion of the likelihood-ratio criterion. 


Consider the third of the three sampling schemes for a 2 X 2 X K table 
discussed in Chapter 4, Section 8, and the two hypotheses 


Hy: A, = o =Ay=1 and H,:A,= c Ay. 


(i) Obtain the likelihood-ratio test statistic for testing H,. 
(ii) Obtain equations that determine the maximum-likelihood estimates of 
the parameters under H,. (These equations cannot be solved explicitly.) 
(iii) Determine the number of degrees of freedom of the limiting x?-distribu- 
tion of the likelihood-ratio criterion for testing (a) H,, (b) H;. 


[For a discussion of these and related hypotheses, see for example Shaffer 
(1973), Plackett (1981), or Bishop, Fienberg, and Holland (1975), and the 
recent study by Liang and Self (1985).] 


Additional Problems 
In generalization of Problem 8 of Chapter 7, let (X,,..., X,,), v = Ll. M 


be independent normal p-vectors with common covariance matrix Z and with 
means 


5 
u-À a, Bj, 
j=1 


where A = (a,j) is a constant matrix of rank s and where the B's are unknown 
parameters. If 6, = Ee; B/^, give explicit expressions for the elements of V and 
S for testing the hypothesis H : 0, = 6, (i = 1,..., p). 


Testing for independence. Let X= (X), i=1,...,p, a — 1,..., N, be à 
sample from a p-variate normal distribution; let q < p, max(q, p — 4) < N; 
and consider the hypothesis H that (Xj,...,X,,) is independent of 
(Xiq+1» +-+ Xip), that is, that the covariances @,; = E(X,; ~ §;)(Xaj — $) 8I 
zero for all i < q, j >q. The problem of testing H remains invariant under 
the transformations X* = X,; + b; and X* = XC, where C is any nonsingu- 
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lar p X p matrix of the structure 


ais Cı 0 
0 C 
with C,, and C» being q X q and (p — q) X (p — 4) respectively. 


(i) A set of maximal invariants under the induced transformations in the 
space of the sufficient statistics X., and the matrix S, partitioned as 


_ (Su Se 
^ (e al 


are the q roots of the equation 
|S,2Sx'Sq - ^Sul = 0- 


(ii) In the case q=1, a maximal invariant is the statistic R= 
5,55; $5/ S1, Which is the square of the multiple correlation coefficient 


between X; and (X5... Xip). The distribution of R? depends only on 
the square p? of the population multiple correlation coefficient, which is 


obtained from R? by replacing the elements of S with their expected 


values 9; ;- 
(ii) Using the fact that the distribution of R? has the density [see for 

example Anderson (1984)] 

que payin Ray mto i ey. 
TE - ITEC - ») 
xe ee S +a) 
h=0 aria - 2) *^l 
and that the hypothesis H for q = lis equivalent to p = 0, show that the 


UMP invariant test rejects this hypothesis when R> O. 


Gv) When p = 0, the statistic 


R? N-p 
TR? eps 


has the F-distribution with p — land N -p degrees of freedom. 


[G): The transformations X* = XC with Cy 
tions 


= I induce on S the transforma- 


(Sirs S2» Sn) > (Si, CuS» CuSnCh) 


with the maximal invariants (Sui, $1555) Application to these invariants 
of the transformations X* — XC with €, = Z completes the proof.] 
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The UMP invariant test of independence in part (ii) of the preceding problem 

is asymptotically robust against nonnormality. 

Bayes character and admissibility of Hotelling’s T 

G) Let(X4,..., Xap) a = L.... n, be a sample from a p-variate normal 
distribution with unknown mean £ = (£;,..., p) and covariance matrix 
X = A-!, and with p < n — 1. Then the one-sample 7?-test of H: = 0 
against K:£ # 0 is a Bayes test with respect to prior distributions Ay 
and A, which generalize those of Chapter 6, Example 13 (continued). 

Gi) The test of part (i) is admissible for testing H against the alternatives 
V? < c for any c > 0. 


[If w is the subset of points (0, 2) of Q,, satisfying E^! = A + y'n for some 
fixed positive definite p X p matrix A and arbitrary = (n,.-.,1,), and 
Q7,» is the subset of points (£, 2) of Qx satisfying Z^! = A + 1/n, & = bE 
for the same A and some fixed b > 0, let Ag and A, have densities defined 
over w and Q, , respectively by 


Ao(n) = GIA + al"? 
and 


b? 
à (n) = GIA + rar em { [a4 5 ^]. 


(Kiefer and Schwartz, 1965).] 
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CHARTERE 


The Minimax Principle 


1. TESTS WITH GUARANTEED POWER 


The criteria discussed so far, unbiasedness and invariance, suffer from the 
disadvantage of being applicable, or leading to optimum solutions, only in 
rather restricted classes of problems. We shall therefore turn now to an 
alternative approach, which potentially is of much wider applicability. 
Unfortunately, its application to specific problems is in general not easy, 
and has so far been carried out successfully mainly in cases in which there 
exists a UMP invariant test. 

One of the important considerations in planning an experiment is the 
number of observations required to insure that the resulting statistical 
procedure will have the desired precision or sensitivity. For problems of 
hypothesis testing this means that the probabilities of the two kinds of 
errors should not exceed certain preassigned bounds, say œ and 1 — f, so 
that the tests must satisfy the conditions 


Eyp(X) <a for 06€Q,, 
(1) 
Ej(X)zB. for 06€Q,. 


If the power function -Ejg( X) is continuous and if a < £, (1) cannot hold 
when the sets Q, and Q, are contiguous. This mathematical difficulty 
corresponds in part to the fact that the division of the parameter values 0 
into the classes Qy and Qg for which the two different decisions are 
appropriate is frequently not sharp. Between the values for which one or the 
other of the decisions is clearly correct there may lie others for which the 
relative advantages and disadvantages of acceptance and rejection are 
approximately in balance. Accordingly we shall assume that is partitioned 
into three sets 


$ - 9,4 9,4 Q,, 
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of which Q, designates the indifference zone, and €, the class of parameter 
values differing so widely from those postulated by the hypothesis that false 
acceptance of H is a serious error, which should occur with probability at 
most 1 — f. 

To see how the sample size is determined in this situation, suppose that 
X,, X,... constitute the sequence of available random variables, and for a 
moment let n be fixed and let X = (X,,..., X,). In the usual applicational 
situations (for a more precise statement, see Problem 1) there exists a test p, 
which maximizes 


Q) inf Eje (X) 


among all level-a tests based on X. Let B, = info, Eg, ( X), and suppose 
that for sufficiently large n there exists a test satisfying (1). [Conditions 
under which this is the case are given by Berger (1951) and Kraft (1955).] 
The desired sample size, which is the smallest value of n for which £, = B. 
is then obtained by trail and error. This requires the ability of determining 
for each fixed n the test that maximizes (2) subject to 


(3) Ejg(X)sa for 6€9j. 


A method for determining a test with this maximin property (of maxi- 
mizing the minimum power over Qx) is obtained by generalizing Theorem : 
of Chapter 3. It will be convenient in this discussion to make a change ie 
notation, and to denote by w and w’ the subsets of 2 previous dent. y 
Qy and Qx. Let P= (Pj, 0 € QU «') be a family of po + 
tions over a sample space (2, 7) with densities py = dP, / dp. wi ae 
to a o-finite measure p, and suppose that the densities pọ(x) one? ae 
functions of the two variables (x, 8) are measurable (4X n and ( z um 
where 4 and 4' are given o-fields over o and o". Under these x al so 
the following theorem gives conditions under which a sind o 
Bayes problem provides a test with the required properties. 

over B and B’, let Pa, be 


Theorem 1. For any distributions A and N 
the most powerful test for testing 


h(x) = f pax) 4&0) 


at level a against 


n(x) = f p(x) 480) 
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and let By. 4; be its power against the alternative h'. If there exist A and A' 
such that 


sup Egg, (X) < a, 


(4) 
inf Espa, (X) = Ban» 


then: 

G) Pa, A: Maximizes inf,,,Eyp(X) among all level-a tests of the hypothe- 
sis H: 0 € w and is the unique test with this property if it is the unique most 
powerful level-a test for testing h against h'. 

(ii) The pair of distributions A, A’ is least favorable in the sense that for 
any other pair v, v' we have 


By nx S B, 


Proof. (i): If ọ* is any other level-a test of H, it is also of level a for 
testing the simply hypothesis that the density of X is h, and the power of 
«* against h’ therefore cannot exceed f, ,.. It follows that 


inf Ejg*(X) < f Eyp*(X) dA(0) < By, y = inf Eopa, (X). 


and the second inequality is strict if p}, a is unique. 
(ii): Let v, v’ be any other distributions over (w, 2) and (w’, 4"), and let 


a(x) = [mx)dr(8), g'(x) = f m) a0). 


Since both p4, y and c, „ are level-a tests of the hypothesis that g(x) is the 
density of X, it follows that 


Brow = fs eG) (3) di(x) = inf Espn, (XO) = Br, w- 


Corollary 1. Let A, A’ be two probability distributions and C a constant 
such that 


1 if f polx) 4N(0) > Cf po(x) 40) 
(5) eG) m iv if f iG) 4A) = Cf p) 4A) 


O if f pol) 40) « C f plx) 4A(0) 
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is a size-a test for testing that the density of X is J,,pg(x) dA(0) and such that 
(6) A(@) = A (a0) = 1, 


where 


Wo = (e: 0€o and Egg, (X) - sup Eyes (0) 
"eo 
e = (o: 0€ and Egg, (X) - jint Enpa, Q0]. 


Then the conclusions of Theorem 1 hold. 


Proof. If h, h', and By, y are defined as in Theorem 1, the assumptions 
imply that q,. a is a most powerful level-a test for testing h against A', that 


sup Espa, (X) = J Eeoa sX) dA (0) =a, 
and that 
inf Espa, w (X) = f Eon (O0 dA'(0) = Ban 


The condition (4) is thus satisfied and Theorem 1 applies. - 
Suppose that the sets Qm Qr and Q, are defined in terms of à 
nonnegative function d, which is a measure of the distance of 6 from H, by 


Q,-(0:4(0)-0), %= (8:0 « d(8) <4), 


2, = (0:4(0) = ^). 


S Si o that the power function of any test is continuous in 6. In 
the limit m = 0, there bab indifference zone. Then Qp becomes the M 
(8: (8) > 0), and the infimum of B(@) over Qy is <a for any Fi -a 
test. This infimum is therefore m: imized by any test satisfying BC ) hs 
for all 0 € Qx, that is, by any unbiased test, so that unbiasedness 1s ca o 
be a limiting form of the imin criterion. A more useful limiting form, 
i i i given by the following 


nimum power locally* if, given 
1964). 


*A different definition of local minimaxity is given by Giri and Kiefer ( 
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any other test q, there exists A, such that 


(y inf B, (0) z inf £, (0) forall 0< A < åp, 
Oy wa 
where w, is the set of 8’s for which d(@) > A. 


2. EXAMPLES 


In Chapter 3 it was shown for a family of probability densities depending 
on a real parameter @ that a UMP test exists for testing H : @ < 6, against 
0 7 0, provided for all 0 < 0' the ratio pẹ(x)/pọ(x) is a monotone 
function of some real-valued statistic. This assumption, although satisfied 
for a one-parameter exponential family, is quite restrictive, and a UMP test 
of H will in fact exist only rarely. A more general approach is furnished by 
the formulation of the preceding section. If the indifference zone is the set of 
0's with 6, < 0 < 6,, the problem becomes that of maximizing the mini- 
mum power over the class of alternatives w: 0 > 0,. Under appropriate 
assumptions, one would expect the least favorable distributions A and A’ of 
Theorem 1 to assign probability 1 to the points 6) and 6,, and hence the 
maximin test to be given by the rejection region Pe (X)/Po (X) > C. The 
following lemma gives sufficient conditions for this to be the case. 


Lemma 1. Let X,..., X, be identically and independently distributed 
with probability density fy(x), where 0 and x are real-valued, and suppose that 
for any 0 < 8" the ratio fy(x)/f(x) is a nondecreasing function of x. Then 
the level-a test œ of H which maximizes the minimum power over w’ is given 
by 


ab, rx C, 
(8) 9(x,...x,)- (Y if r(x,...,x,) = C 
Of Sele 5.15 e )*« €, 


where r(xy,..., Xn) = fo Q0)... fo Xn)/fog(X1) -+ - fo (Xn) and where C and 
y are determined by : 


(9) E,@(X,,--., X,) =a. 


Proof. The function g(x,,..., x,,) is nondecreasing in each of its argu- 
ments, so that by Lemma 2 of Chapter 3 


Ejg( Xy,..., X,) < Epp ( Xy,..., Xn) 


‘properties of such densities 
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when 0 < 6’. Hence the power function of o is monotone and ọ is a level-a 
test. Since p = Pa, a» Where A and A’ are the distributions assigning 
probability 1 to the points &j and 6,, the condition (4) is satisfied, which 
proves the desired result as well as the fact that the pair of distributions 
(A, A’) is least favorable. 


Example 1. Let @ be a location parameter, so that fy(x) = g(x — 0), and 
suppose for simplicity that g(x) > 0 for all x. We will show that a necessary and 
sufficient condition for f; (x) to have monotone likelihood ratio in x is that —log g 
is convex. The condition of monotone likelihood ratio in x, 


_ 9! 6 
FG i RR forall x«x', 06«6, 
g(x-0) g(x’—9) 


is equivalent to 
log g(x’ — 0) + log g(x — 9) < log g(x — 8) + log g(x’ = 8^). 


— t(x’ — 0) and x'-0 2 -10(x-0)* 
j X E + p- 6), a sufficient condition for this to 
see that this condition is also 
_ 6’ =a, x'—-0=b, and 
1(a + b), and the condition 


Since x — 0 = t(x — 0) + ( 
t(x' — 0), where t = (x’ — x)/(*' - 
hold is that the function —logg is convex. To 
necessary, let a < b be any real numbers, and let x 
yb 2x — 8. Then. x= 8 =e Ot ie) 
of monotone likelihood ratio implies 


2[log g(a) + log g(4)] < log g[4(a + b)]. 

—log g is convex.” 

ed strongly unimodal. Basic 
i i . Strong 

were obtained by Ibragimov (1956) 

f total positivity. A density of the form 

g(x — 0) which is totally positive of order r is said to be a Polya frequency 


function of order r. It follows from Example 1 that g(x — 8) is a Polya 


i i if it is strongly unimodal. [For 
frequency function of order 2 if and x m : Ped ay ieee 


further results concerning Polya Tr (1979), Huang and Ghosh 


Since log g is measurable, this in turn implies that 


A density g for which — log g is convex is call 


unimodality is a special case © 


densities, see Karlin (1968), Marshall and Olkin 
(1982), and Loh (1984a, b) ] ition [besides the normal 
a. j «fv the above condition [besi 
Two dis sapere aie densities po(Xv 7 x,) form an 


distribution, for which 


istribution with 
exponential family] are the distributio 


double exponential 


g(x) =i 


*See Sierpinski (1920). 
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and the logistic distribution, whose cumulative distribution function is 


G(x) = 


> 


Let Bae 
so that the density is g(x) = e~*/(1 + e-*)?. 


Example 2. To consider the corresponding problem for a scale parameter, let 
fo(x) = 0^ !h(x/0) where h is an even function. Without loss of generality one may 
then restrict x to be nonnegative, since the absolute values | X;|,..., | X,| form a set 
of sufficient statistics for 0. If Y, = log X, and x = log 6, the density of Y, is 


h(e7*)e-7. 


By Example 1, if h(x) > 0 for all x > 0, a necessary and sufficient condition for 
Jo (x)/fo(x) to be a nondecreasing function of x for all @ < 0’ is that —log[ e"h(e?)] 
or equivalently —log h(e”) is a convex function of y. An example in which this 
holds—in addition to the normal and double-exponential distributions, where the 
resulting densities form an exponential family—is the Cauchy distribution with 


7 Twill 
Som 


Since the convexity of — log h( y) implies that of — log A(e"), it follows that if h 
is an even function and h(x — 0) has monotone likelihood ratio, so does ^(x/0). 
When A is the normal or double-exponential distribution, this property of ^(x/0) 
follows therefore also from Example 1. That monotone likelihood ratio for the 
scale-parameter family does not conversely imply the same property for the associ- 
ated location parameter family is illustrated by the Cauchy distribution. The 
condition is therefore more restrictive for a location than for a scale parameter. 


The chief difficulty in the application of Theorem 1 to specific problems 
is the necessity of knowing, or at least being able to guess correctly, a pair of 
least favorable distributions (A, A’). Guidance for obtaining these distribu- 
tions is sometimes provided by invariance considerations. If there exists a 
Broup G of transformations of X such that the induced group G leaves both 
w and o' invariant, the problem is symmetric in the various 0's that can be 
transformed into each other under G. It then seems plausible that unless A 
and A’ exhibit the same symmetries, they will make the statistician's task 
easier, and hence will not be least favorable. 


Example 3. In the problem of paired comparisons considered in Example 7 of 
Chapter 6, the observations X, (i — 1,..., n) are independent variables taking on 
the values 1 and 0 with probabilities p, and q, = 1 — p,. The hypothesis H to be 
tested specifies the set o : max p, < 1. Only alternatives with p; > } for all i are 
considered, and as w’ we take the subset of those alternatives for which max p; = > 
+ô. One would expect A to assign probability 1 to the point p; = ::: p, = 1, and 
A’ to assign positive probability only to the n points ( p,,..., p,) which have n — 1 
coordinates equal to } and the remaining coordinate equal to 1 + 8. Because of the 
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symmet with regard to the n variables, it seems plausibl hould assign 

Ty ur " e that A’ i 

equal probability 1/n to each of these n points. With these choices the r^ ER 
id AC 


rejects when 
T E 
Y | l4 d a 
1 i 


2 


This is equivalent to 


n 
YxC, 
i-l 
which had previously been seen to be UMP invariant for this problem. Since the 
critical function qx. (Xj... Xn) i. nondecreasing in each of its arguments, it 
follows from Lemma 2 of Chapter 3 that p, < pj fori=1,.... implies 


E, pr. Os Xn) Ed. MOTO) 


and hence the conditions of Theorem 1 are satisfied. 
Example 4. Let X = (X,,..., X,) bea sample from N(£, o?), and consider the 
t the set of alternatives w’: 0 < 9; OF 9 2 0; 


(0, < ay < 0). This problem remains invariant under the transformations X - 
induce the group G of transformations 


E = £ + c, o' = ø. One would therefore expect the least favor 


implies that A assigns to any interval a measure proportional to 
interval. Hence A cannot be a probability measure and Theorem 1 is not directly 
applicable. The difficulty can be avoided by approxima 
probability distributions, in the presen 
distributions N(0, k), k = L,2,-.. - 
under consideration, it happens that there also exist 
d A^, which are true probability distributions and 
distributions can be obtained i 
i 3, Section 9, as follows. On v, 
£ is taken as the normal 
£, and with variance (o2 — aj)/n. Under A’ all 
T ili n the two lines a = 9, ani 
Probability y hips ur x nal distribution with mean 
£, and variance (02 — o2)/n, while A^, assigns p! 0 
computation ee t/a carried out in Chapter 3, Section 9, then shows 


acceptance region to be given by 


p xs E 
0 zs TN a] 


« C, 
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which is equivalent to 


C,<¥(x,-%) <G. 


The probability of this inequality is independent of £, and hence C, and C, can be 
determined so that the probability of acceptance is 1 — a when o = og, and is equal 
for the two values c = o, and o = o;. 

It follows from Section 7 of Chapter 3 that there exist p and C which lead to 
these values of C, and C, and that the above test satisfies the conditions of 
Corollary 1 with w =, and with «( consisting of the two lines o = o, and 
o = 0. 


3. COMPARING TWO APPROXIMATE HYPOTHESES 


As in Chapter 3, Section 2, let P) # P, be two distributions possessing 
densities py and p, with respect to a measure p. Since distributions even at 
best are known only approximately, let us assume that the true distributions 
are approximately P, or P, in the sense that they lie in one of the families 


(10) %,={02:0=(1-¢)P,+¢G,}, | i-0,1, 


With €o, e; given and the G, arbitrary unknown distributions. We wish to 

find the level-a test of the hypothesis H that the true distribution lies in 2%, 

which maximizes the minimum power over 2^. This is the problem consid- 

aed in Section 1 with @ indicating the true distribution, Q, = %, and 
k7 4. 

The following theorem shows the existence of a pair of least favorable 
distributions A and A’ satisfying the conditions of Theorem 1, each 
assigning probability 1 to a single distribution, A to Q) € P, and A’ to 
Q, € P., and exhibits the Q, explicitly. 


Theorem 2. Let 
i (ü-&)p(x) if a a, 
q(x) = aa , A, 
(1) 
oa M 
ina po(x) 
TUO TU apice. 


px) a. 
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(i) For all 0 < e;< 1, there exist unique constants a and b such that qo 
and q, are probability densities with respect to p; the resulting q; are 
members of P, (i = 0,1). 

(ii) There exist 85, Ô, such that for all e; < 8, the constants a and b satisfy 
a < b and that the resulting qo and q, are distinct. 

(iii) Ife; 8, fori — 0,1, the families P) and P, are nonoverlapping and 
the pair (qo, qı) is least favorable, so that the maximin test of Po 
against P, rejects when qi(x)/qo(x) is sufficiently large. 

Note. Suppose a « b, and let 


Then 


ka when r(x) <a, 
(12) r*(x) =( kr(x) when a«r(x) <5, 
kb when b< r(x). 


The maximin test thus replaces the original probability ratio with a censored version. 


Proof. The proof will be given under the simplifying assumption that 
the sample space. 


Po(x) and p,(x) are positive for all x in f 4 
1 (i): For A to be a probability density, a must satisfy the equation 


1 
(13) P,[r(X) > a] + aPy[r(X) sad = Fe 


If (13) holds, it is easily checked ‘that d; € 9, (Problem 10). To prove 
existence and uniqueness of a solution a of (13), let 


y(c) = Pylr(X) > el + cP)[r(X) < ¢]- 


Then 
(14) vio) =P andy, POY Oe Pr; 
Furthermore (Problem 12) 


(15) y(c + A) - (e) = af, nC? du(x) 


mat a Ax] a" BR 


514 THE MINIMAX PRINCIPLE [9.3 


It follows from (15) that 0 € y(c + A) — y(c) < A, so that y is continuous 
and nondecreasing. Together with (14) this establishes the existence of a 
solution. To prove uniqueness, note that 


(16) vet A)—r(c) = Af po(x) d(x) 


and that y(c) = 1 for all c for which 
(17) P[r(x)<c]=0 (i=0,1). 


If co is the supremum of the values for which (17) holds, (16) shows that y 
is strictly increasing for c > cy and this proves uniqueness. The proof for b 
is exactly analogous (Problem 11). 

(ii): As e, > 0, the solution a of (13) tends to cy. Analogously, as 
€; > 0, b > co (Problem 11). 

(iii): This will follow from the following facts: 


(a) When X is distributed according to a distribution in 2, the 
statistic r*( X) is stochastically largest when the distribution of X is 
Qo. 

(b) When X is distributed according to a distribution in 2}, r*( X) is 
stochastically smallest for Q,. 

(c) r*(X) is stochastically larger when the distribution of X is Q, than 
when it is Qù. 


These statements are summarized in the inequalities 
(18) 
Oslr*(X) < d = Oolr*(X) < d = obo < d > edit 00 <4] 


for all ¢ and all Q; € 2. 

From (12), it is seen that (18) is obvious when t < ka or t > kb. Suppose 
therefore that ak < t < bk, and denote the event r*(X) < t by E. Then 
Q6(E) = (1 — e)P,(E) by (10). But r*(x) « t € kb implies r(X) < b 
and hence Q (E) = (1 — e,)P (E). Thus Q4(E) > Q,(E), and analo- 
gously Qí(E) < Q,(E). Finally, the middle inequality of (18) follows from 
Corollary 1 of Chapter 3. 

If the e's are sufficiently small so that Qo * Q,, it follows from (a)-(c) 
that Py and P, are nonoverlapping. 

That (Qo, Q) is least favorable and the associated test y is maximin now 
follows from Theorem 1, since the most powerful test p for testing Qo 
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against Q, is a nondecreasing function of q,( X)/qo( X). This shows that 
E(X) takes on its sup over Pp at Qo and its inf over 2, at Q}, and this 
completes the proof. 

Generalizations of this theorem are given by Huber and Strassen (1973, 
1974). See also Rieder (1977) and Bednarski (1984). An optimum permuta- 
tion test, with generalizations to the case of unknown location and scale 
parameters, is discussed by Lambert (1985). 

When the data consist of n identically, independently distributed ran- 
dom variables X,,..., Xn the neighborhoods (10) may not be appropriate, 
since they do not preserve the assumption of independence. If P, has 
density 


(19) PHOBMRCALI COSS C OD 


a more appropriate model approximating (19) may then assign to X= 
(X,,..., X,) the family 2; of distributions according to which the X, are 
independently distributed, each with distribution 


(20) (iiss e) (xj) + «G;(x;) 


where F, has density f; and where as before the G; are arbitrary. 


ith x = x; satisfy (18) 

Corollary 2. Suppose qo and q, defined by (11) wit j | 
and hence are a least favorable pair for testing Po against Pı on te Fo d 
the single observation Xj. Then the pair of distributions wit F ie. 
qi(x1)- <- q(x,) G = 0,1) is least favorable for testing Pg against Pf, so 


that the maximin test is given by 


1 1 Il q(x) | Š » 


Q) — et = ( ito Mie 


5 = e 

Proof. By assumption, the random variables Y, = e 2 " 

stochastically increasing as one move successively from a "y, which 

Q, to Q; € P,. The same is then true of any function ay EE nae 
is nondecreasing in each of its arguments by nar mS Theorem 2i 

hence of p defined by (21): e qeu E es 2 consider now the 

Instead of the problem of testing fo së ested against 0 > 9, 


be tes 
ituati here H: 0 < % is t0 © ad 
situation of Lemmi aE dent observations X,, each distributed 


is of n indepen : ; 
ae | Fy(x,) whose density fo(x;) 18 assumed to have 


monotone likelihood ratio in Xj- 
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A robust version of this problem is obtained by replacing F} with 
(22) (-e)5(x)-«eG(x) f= ,...,0, 


where e is given and for each 6 the distribution G is arbitrary. Let P** 
and Pf* be the classes of distributions (22) with @ < 6 and 6 > 6, 
respectively; and let Z* and 2 be defined as in Corollary 2 with fo, in 
place of f, Then the maximin test (21) of P% against #* retains this 
property for testing AF* against Pf*. 

This is proved in the same way as Corollary 2, using the additional fact 
that if Fj, is stochastically larger than Fg, then (1 — «) Fy, + €G is stochasti- 
cally larger than (1 — €)F, + «G. 


4. MAXIMIN TESTS AND INVARIANCE 


When the problem of testing Q, against €, remains invariant under a 
certain group of transformations, it seems reasonable to expect the existence 
of an invariant pair of least favorable distributions (or at least of sequences 
of distributions which in some sense are least favorable and invariant in the 
limit), and hence also of a maximin test which is invariant. This suggests the 
possibility of bypassing the somewhat cumbersome approach of the preced- 
ing sections. If it could be proved that for an invariant problem there always 
exists an invariant test that maximizes the minimum power over Qx, 
attention could be restricted to invariant tests; in particular, a UMP 
invariant test would then automatically have the desired maximin property 
(although it would not necessarily be admissible). These speculations turn 
out to be correct for an important class of problems, although unfortunately 
not in general. To find out under what conditions they hold, it is convenient 
first to separate out the statistical aspects of the problem from the group- 
theoretic ones by means of the following lemma. 


Lemma 2. Let P= {P}, 0 € Q} be a dominated family of distributions 
on (2, of), and let G be a group of transformations of (X, s), such that the 
induced group G leaves the two subsets Q jg and Qy of Q invariant. Suppose 
that for any critical function @ there exists an (almost) invariant critical 
function v) satisfying 


Q3) in Ege(X) < Epp (X) < supEgg(X) 
G 


for all 0 & Q. Then if there exists a level-a test g, maximizing infa, Egg X). 
there also exists an (almost) invariant test with this property. 
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Proof. Let info, Eepo(X) = B, and let jy be an (almost) invariant test 
such that (23) holds with o = go, y = y. Then 


Ej (X) < supEggo(X) <a forall 0 EQ; 
Ic 


and 


Egho(X) = inf Eggo( X) >B forall 6€Qy,, 


as was to be proved. 

To determine conditions under which there exists an invariant or almost 
invariant test y satisfying (23), consider first the simplest case that G is a 
finite group, G = { gı, ---» y) say. If y is then defined by 


1 N 
(24) V(x) =5 X olg), 


it is clear that y is again a critical function, and that it is invariant under G. 


It also satisfies (23), since Egg(gX) = Egg ( X) so that EjJ( X) is the 
average of a number of terms of which the first and last member of (23) are 
the minimum and maximum respectively. 

An illustration of the finite case is furnished by Example 3. Here the 
problem remains invariant under the n! permutations of the variables 
(X,,..., X,). Lemma 2 is applicable and shows that there exists an in- 
variant test maximizing infg, Ejg( X). Thus in particular the UMP gii 
test obtained in Example 7 of Chapter 6 has this maximin property an 


th i lution of the problem. i 
erefore constitutes a sotu! li of culis] (x) M6 


The definition (24) suggests the possibi i l 
other cases by averaging the values of g(gx) with ne end 
probability distribution over the group G. To see what con oe ng 
required of this distribution, let J bea o-field of d iria 
probability distribution over (G, 4). Disregarding measurability Pp! 


for the moment, let y be defined by 
Q5) v(x) = [o(gx) (2. 


i d by applying 
Then 0 < y <1, and (23) is seen to hol i 
(Theorem 3 of Chapter 2) to the integral of V with respec 


tion P,. For any go € € 


y (gox) = [( 80x 


Fubini’s theorem 
t to the distribu- 


) dv(g) = fo hx) 4*0) 
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where h = gg, and where v* is the measure defined by 
»*(B)-»(Bg;! forall BE &, 


into which » is transformed by the transformation h = ggo. Thus y will 
have the desired invariance property, J/(gox) = (x) for all go € G, if v is 
right invariant, that is, if it satisfies 


(26) »(Bg)-»(B) forall BEB, geG. 


The measurability assumptions required for the above argument are: 
(i) For any A E€., the set of pairs (x, g) with gx € A is measurable 
(2 X B). This insures that the function defined by (25) is again measur- 
able. (ii) For any B € 4, g € G, the set Bg belongs to 4. 


Example 5. If G isa finite group with elements g,,..., gy, let 2 be the class of 
all subsets of G and » the probability measure assigning probability 1/N to each of 
the N elements. The condition (26) is then satisfied, and the definition (25) of y in 
this case reduces to (24). 


Example 6. Consider the group G of orthogonal n X n matrices T, with the 
group product T,T, defined as the corresponding matrix product. Each matrix can 
be interpreted as the point in n?-dimensional Euclidean space whose coordinates are 
the n? elements of the matrix. The group then defines a subset of this space; the 
Borel subsets of G will be taken as the o-field Z. To prove the existence of a right 
invariant probability measure over (G, #),* we shall define a random orthogonal 
matrix whose probability distribution satisfies (26) and is therefore the required 
measure. With any nonsingular matrix x = (x,,), associate the orthogonal matrix 
y= f(x) obtained by applying the following Gram-Schmidt orthogonalization 
process to the n row vectors x, = (x;,..., Xin) of x: y, is the unit vector in the 
direction of x; y, the unit vector in the plane spanned by x, and x;, which is 
orthogonal to y; and forms an acute angle with x;; and so on. Let y = (y,;) be the 
matrix whose ith row is y,. 

Suppose now that the variables X,; (i, j = 1,..., n) are independently distrib- 
uted as N(0,1), let X denote the random matrix (X,,), and let Y = f(X). To show 
that the distribution of the random orthogonal matrix Y satisfies (26), consider any 
fixed orthogonal matrix T and any fixed set B € 4. Then P(Y € BT) = P(YT' € 
B) and from the definition of f it is seen that YI’ = f( XT"). Since the n? elements 
of the matrix XT" have the same joint distribution as those of the matrix X, the 
matrices f( XT”) and f(X) also have the same distribution, as was to be proved. 


` Examples 5 and 6 are sufficient for the applications to be made here. 
General conditions for the existence of an invariant probability measure, of 
which these examples are simple special cases, are given in the theory of 
Haar measure. [This is treated, for example, in the books by Halmos (1974), 


*A more detailed discussion of this invariant measure is given by James (1954). 


9.5] THE HUNT-STEIN THEOREM 519 


Loomis (1953), and Nachbin (1965). For a discussion in a statistical setting, 
see Eaton (1983), Farrell (1985), and for a more elementary treatment 
Berger (1985).] 


5. THE HUNT-STEIN THEOREM 


Invariant measures exist (and are essentially unique) for a large class of 
groups, but unfortunately they are frequently not finite and hence cannot be 
taken to be probability measures. The situation is similar and related to that 
of the nonexistence of a least favorable pair of distributions in Theorem 1. 
There it is usually possible to overcome the difficulty by considering instead 
a sequence of distributions, which has the desired property in the limit. 
Analogously we shall now generalize the construction of y as an average 
with respect to a right-invariant probability distribution, by considering a 
sequence of distributions over G which are approximately right-invariant 
for n sufficiently large. Y 

Let P= ( Py, 0 € Q) bea family of distributions over a Euclidean space 
(£, sf) dominated by a o-finite measure p, and let G be a group of 
transformations of (Z, ) such that the induced group G leaves Q in- 


variant. 


Theorem 3. (Hunt-Stein.) Let B be 
any A € xf the set of pairs (x, 8) with 
B € Band g € G the set Bg is in B. Suppose 
probability distributions v, over (G, B) which is asymp: 
in the sense that for any g € G, BE a 


a o-field of subsets of G such that for 

gx EA is in SXB and for any 
that there exists a sequence of 
totically right-invariant 


(27) lim |n (35) - (01-9 


Then given any eritical function 9, there exists a critical function y which is 


almost invariant and satisfies (23). 
Proof. Let 
p(x) = [o(e n (Q2. 
i i tween 0 and 1. By the weak compact- 
which as before is measurable and Padi) there exists a subsequence (Yn) 


ness theorem (Theorem 3 of the Ap] xists 
and a measurable function y between 0 and 1 satisfying 


lim fV, pdn 7 [vns 
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for all p-integrable functions p, so that in particular 
lim Eyy,,(X) = Eoy(X) 
ico 


for all 0 € Q. By Fubini's theorem 


Epi, (X) = [[Eo9(sX)] dv, (8) = f Ego (X) dv, (2) 
so that ; 
inf Ep (X) < Eg, (X) < supEge (X), 
and y satisfies (23). 


In order to prove that is almost invariant we shall show below that for 
all x and g, 


(28) Vu (gx) — v. (x) > 0. 


Let 1,(x) denote the indicator function of a set A € 7. Using the fact that 
T,4( 8x) = L,(x), we see that (28) implies 


J p(x) dPo(x) 


Jim. fV, (x) LG) dP p(x) 


lim fyn (9x) Cx) dPo(x) 


ll 


J )Igalx) dal x) = f$ (ux) devis) 
and hence (gx) = (x) (a.e. P), as was to be proved. 


To prove (28), consider any fixed x and any integer m, and let G be 
partitioned into the mutually exclusive sets 


1 
B= [he Grm <col) <a, +2}, k=0,...,m, 
m 


where 4, = (k — 1)/m. In particular, B, is the set {h E G: (hx) = 0). It 
is seen from the definition of the sets B, that 


DELE E [09 dv, (h) < D (ax + 5. Ja 


=0 


m 1 
< XM aa (B) + — 
k-0 ` m 
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and analogously that 


m m 

1 
E fy pt O) dr 0) — Yam (Bu) s 
from which it follows that 


Iv. (e) - | s End] (7) = (B+ E- 


By (27) the first term of the right-hand side tends to zero as i tends to 
infinity, and this completes the proof. 

When there exist a right-invariant measure v over G and a sequence of 
subsets G, of G with G, € G,,,, UG, = G, and »(G,) = c, < %, it is 
suggestive to take for the probability measures », of Theorem 3 the 
measures v/c, truncated on G,. This leads to the desired result in the 
example below. On the other hand, there are cases in which there exists such 
a sequence of subsets of G, but no invariant test satisfying (23) and hence 


no sequence v, satisfying (27). 


Example 7. Let x = (Xy, Xn) af be the class of Borel sets in n-space, and 
G the group of translations (xi + 8)---»¥n + 8), ~% < g < co. The elements of 
G can be represented by the real numbers, and the group product gg’ is then the 
sum g +g’. If @ is the class of Borel sets on the real line, the measurability 
assumptions of Theorem 3 are satisfied. Let v be Lebesgue measure, which is clearly 
invariant under G, and define », to be the uniform distribution on the interval 
I(-n,n) = {gi -nsgs n). Then for all Bed,geG, 


1 lel 

|», (B) - n (Bg) |= 5-1 B A Cn I -»[BnI(-n- n - BIS 9," 
hi poet 

oT OT of scale transformations (ax,,--+) Xo) 


This argument also covers the group 
0 « a < co, which can be transformed into 
rithms. 


When applying the Hunt-Stein th 
tests, it is frequently convenient to carry out the V 
done in Theorem 7 of Chapter 6. Suppose that the problem remains 
invariant under two groups D and E, and denote by y = s(x) a maximal 


invari i D and by E* the group defined in Theorem 2, 
mo in jac If D and E* satisfy the conditions 
of the Hunt-Stein theorem, it follows first that there exists a maximin test 
depending only on y — s(x), and then that there Tem maximin test 
depending only on à :mal invariant z = t) under E*. 


the translation group by taking loga- 


eorem to obtain invariant minimax 
calculation in steps, as was 
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Example 8 Consider a univariate linear hypothesis in the canonical form in 
which Y,,..., Y, are independently distributed as N(,, 07), where it is given that 
Tei = co = n, = 0, and where the hypothesis to be tested is n, = --- =n, = 0. 
It was shown in Section 1 of Chapter 7 that this problem remains invariant under 
certain groups of transformations and that with respect to these groups there exists a 
UMP invariant test. The groups involved are the group of orthogonal transforma- 
tions, translation groups of the kind considered in Example 7, and a group of scale 
changes. Since each of these satisfies the assumptions of the Hunt-Stein theorem, 
and since they leave invariant the problem of maximizing the minimum power over 
the set of alternatives 


r 2 
(29) Lee (9. 


it follows that the UMP invariant test of Chapter 7 is also the solution of this 
maximin problem. It is also seen slightly more generally that the test which is UMP 
invariant under the same groups for testing 


r 2 
Er 

Deen 
2 0 

i219 


(Problem 4 of Chapter 7) maximizes the minimum power over the alternatives (29) 
for yo < yy. 


, Example 9. (Stein) Let G be the group of all nonsingular linear transforma- 
tions of p-space. That for p 1 this does not satisfy the conditions of Theorem 3 is 
shown by the following problem, which is invariant under G but for which the UMP 
invariant test does not maximize the minimum power. Generalizing Example 1 of 
Chapter 6, let X-(X,..., X), Y= (Y,,..., ¥,) be independently distributed 
according to p-variate normal distributions with zero means and nonsingular 
covariance matrices E(X,.X)) = o, and E(Y.Y) = Ac,,, and let H:A < Ao be 
tested against A > A, (A, < Ay), the 9,; being unknown. 

This problem remains invariant if the two vectors are subjected to any common 
nonsingular transformation, and since with probability 1 this group is transitive over 
the sample space, the UMP invariant test is trivially p(x, y) = a. The maximin 
power against the alternatives A > A, that can be achieved by invariant tests is 

therefore a. On the other hand, the test with rejection region Y?/X? > C has a 
strictly Increasing power function B(A), whose minimum over the set of alternatives 
A > A, is B(A,) > B(A) =a. 


It is a remarkable feature of Theorem 3 that its assumptions concern only 


the gr oup G and not the distributions P,.* When these assumptions hold for 
a certain G it follows from (23) as in the proof of Lemma 2 that for any 


*These assumptions are essentially equivalent to the condition that the group G is 
amenable. Amenability and its relationship to the Hunt-Stein theorem are discussed by Bondar 
and Milnes (1982) and (with a different terminology) by Stone and von Randow (1968). 
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testing problem which remains invariant under G and possesses a UMP 
invariant test, this test maximizes the minimum power over any invariant 
class of alternatives. Suppose conversely that a UMP invariant test under G 
has been shown in a particular problem not to maximize the minimum 
power, as was the case for the group of linear transformations in Example 9. 
Then the assumptions of Theorem 3 cannot be satisfied. However, this does 
not rule out the possibility that for another problem remaining invariant 
under G, the UMP invariant test may maximize the minimum power. 
Whether or not it does is no longer a property of the group alone but will in 
general depend also on the particular distributions. 

Consider in particular the problem of testing H: $= + =§, =00n 
the basis of a sample (Xa: -> Xap), & = 1,---, M, from a p-variate normal 
distribution with mean E(X,;) = £; and common covariance matrix (0;;) = 
(a,,)~*. This was seen in Section 3 of Chapter 8 to be invariant under a 
number of groups, including that of all nonsingular linear transformations 
of p-space, and a UMP invariant test was found to exist. An invariant class 


of alternatives under these groups is 


(30 py ev 


and the question whether the T?-test 


Here Theorem 3 is not applicable, e 
power over the alternatives 


of H : y = 0 maximizes the minimum 
(31) YXajt£; =v 
rmidable difficul- 


[and hence a fortiori over the alternatives (30)] presents fo ers 
ties. The minimax property was proved for the case p = 2, n= T y mm 
Kiefer, and Stein (1963), for the case p — 2, n = 4 by Linnik, Pliss, 


ii - by Salaevskii (1971). The 
Salaevskii (1968), and for p = 2 and all n 2 3 ilaevsi 
proof is xir by first reducing the problem through invariance under the 


i i licable, 
oup G, of Example 11 of Chapter 6, to which Theorem 3 is app! 
p tcs appa Tuin 1 to the reduced problem. It H ig ite 
this approach that it also establishes the admissibility of afa ua 
against the alternatives (31). In view of the inadmissibility . M aae 
estimation when p = 3 (see TPE, Sections 4.5 and 4.6), i mare pry 
that T? is admissible for p = 3, and hence that the same 


ini in this situation. 
the minimax property 1n this situatior . f 
Te ER becomes much easier when the minimax property is consid 


ered against local or distant alternatives rather than against (31). Precise 
definitions and proofs of the fact that T? possesses 


these properties for all P 
and n are provided by Giri and Kiefer (1964) and in the references given 1n 
Chapter 8, Section 3. 
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The theory of this and the preceding section can be extended to con- 
fidence sets if the accuracy of a confidence set at level 1 — a is assessed by 
its volume or some other appropriate measure of its size. Suppose that the 
distribution of X depends on the parameters 0 to be estimated and on 
nuisance parameters 9, and that ji is a o-finite measure over the parameter 
set w = (0:(0, 9) € Q}, with w assumed to be independent of 9. Then the 
confidence sets S( X) for 0 are minimax with respect to p at level 1 — a if 
they minimize 

sup Ey a n [S( X)] 


among all confidence sets at the given level. 

The problem of minimizing Eu[S( X)] is related to that of minimizing 
the probability of covering false values (the criterion for accuracy used so 
far) by the relation (Problem 26) 


(32) Eo, oM (S(X)] =f Pon [0 € SQO] an(9). 


which holds provided p assigns measure zero to the set (0 = )}. (For the 
special case that @ is real-valued and u Lebesgue measure, see Problem 29 
of Chapter 5.) 

Suppose now that the problem of estimating @ is invariant under a group 
G in the sense of Chapter 6, Section 11 and that p satisfies the invariance 
condition 


(33) n[SCgx)] = n[S(x)]. 


If uniformly most accurate equivariant confidence sets exist, they minimize 
(32) among all equivariant confidence sets at the given level, and one may 
hope that under the assumptions of the Hunt-Stein theorem, they will also 
be minimax with respect to u among the class of all (not necessarily 
equivariant) confidence sets at the given level. Such a result does hold and 
can be used to show for example that the most accurate equivariant 
confidence sets of Examples 17 and 18 of Chapter 6 minimize their 
maximum expected Lebesgue measure. A more general class of examples is 
provided by the confidence intervals derived from the UMP invariant tests 
of univariate linear hypotheses such as the confidence spheres for 6; = p + &; 
or for o; given in Section 5 of Chapter 7. 

Minimax confidence sets S(x) are not necessarily admissible; that is, 
there may exist sets S’(x) having the same confidence level but such that 


Eo gulS(X)] < Es n[S(X)] forall 0, ð 


with strict inequality holding for at least some (0, 9). 
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Example 10. Let X, (i= 1,..., s) be independently normally distributed with 
mean E(X,) = 6, and variance 1, and let G be the group generated by translations 
X, + c, (i= 1,...,s) and orthogonal transformations of (X,,..., X,). (G is the 
Euclidean group of rigid motions in s-space.) A slight generalization of Example 17 
of Chapter 6 shows the confidence sets 


(34) Y(6- XY sc 


to be uniformly most accurate equivariant. The volume p[S(X)] of any confidence 
set S( X) remains invariant under the transformations g € G, and it follows from 
the results of Problems 30 and 31 and Examples 7 and 8 that the confidence sets 
(34) minimize the maximum expected volume. However, very surprisingly, they are 
not admissible unless s — 1 or 2. This result, which will not be proved here, is 
closely related to the inadmissibility of X,,..., X, asa point estimator of (6... 6) 
for a wide variety of loss functions. The work on point estimation, which is 
discussed in TPE, Sections 4.5 and 4.6, for squared error loss, provides an easier 
access to these ideas than the present setting, A convenient entry into the literature 
on admissibility of confidence sets is Hwang and Casella (1982), NO: 

The inadmissibility of the confidence sets (34) is particularly surprising 1n that 
the associated UMP invariant tests of the hypotheses H: 6, — 6, (i= 1... 5) are 
admissible (Problems 28, 29). 


6. MOST STRINGENT TESTS 


the consideration of tests that maximize 


the minimum power over a class Qx of alternatives is the determination * 

an appropriate Q9. If no information is available on which to base t x 

choice of this set and if a natural definition is not imposed by invarian 
finition can be given in terms of the 


arguments, a frequently reasonable del r e 
pov that can be achieved against the various alternatives. The envelope 
6, Problem 15, by 


power function By was defined in Chapter 
pz (0) = sup B, (9). 


One of the practical difficulties in 


ere the supremum is taken 


d wh 
where 8, denotes the power of a test p an e a 
over all level-a tests of H. Thus B*(0) is the maximum power that ca 


attained at level a against the alternative 0. gir it ea E ky follows 
under mild restrictions from Theorem 3 of the App À 


st- (0: B (0) = ^) 


i loser to 
then of two alternatives 9, € Sip 6, 7 rie isa o 
idi ay than 1 Aog à 
e Wn bee us dime of an alternative from he in ain x 
the available information has been encountered before. If for p 
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X,,..., X, is a sample from N(£, o”), the problem of testing H : £ « 0 was 
discussed (Chapter 5, Section 2) both when the alternatives £ are measured 
in absolute units and when they are measured in o-units. The latter 
possibility corresponds to the present proposal, since it follows from invari- 
ance considerations (Problem 15 of Chapter 6) that B7(£, a) is constant on 
the lines £/a — constant. 

Fixing a value of A and taking as Qx the class of alternatives 0 for which 
B*(8) = A, one can determine the test that maximizes the minimum power 
over Qg. Another possibility, which eliminates the need of selecting a value 
of A, is to consider for any test p the difference 97(0) — B,(@). This 
difference measures the amount by which the actual power 8,() falls short 
of the maximum power attainable. A test that minimizes 


(35) sup AOR B,(4)] 


is said to be most stringent. Thus a test is most stringent if it minimizes its 
maximum shortcoming. 

Let p, be a test that maximizes the minimum power over S;*, and hence 
minimizes the maximum difference between 8*(0) and B,(@) over Sx. If qa 
happens to be independent of A, it is most stringent. This remark makes it 
possible to apply the results of the preceding sections to the determination 
of most stringent tests. Suppose that the problem of testing H:9¢w 
against the alternatives € Q — w remains invariant under a group G, that 
there exists a UMP almost invariant test pọ with respect to G, and that the 
assumptions of Theorem 3 hold. Since (8) and hence the set Sj is 
invariant under G (Problem 15 of Chapter 6), it follows that pọ maximizes 
the minimum power over S for each A, and gy is therefore most stringent. 

As an example of this method consider the problem of testing 
H: py,..., p, S 3 against the alternative K : p, > } for all i, where p; is 
the probability of success in the ith trial of a sequence of n independent 
trials. If X, is 1 or 0 as the ith trial is a success or failure, then the problem 
remains invariant under permutations of the X's, and the UMP invariant 
test rejects (Example 7 of Chapter 6) when EX,» C. It now follows from 
the remarks above that this test is also most stringent. 

Another illustration is furnished by the general univariate linear hypothe- 
sis. Here it follows from the discussion in Example 8 that the standard test 
for testing H:m = +- =n, =0 or H’:L'_yn2/o? < 4} is most strin- 
gent. 

When the invariance approach is not applicable, the explicit determina- 
tion of most stringent tests typically is difficult. The following is a class of 
problems for which they are easily obtained by a direct approach. Let the 
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distributions of X constitute a one-parameter exponential family, the den- 
sity of which is given by (12) of Chapter 3, and consider the hypothesis 
H: 0 = 0. Then according as 0 > 6, or 0 < %, the envelope power B*(0) 
is the power of the UMP one-sided test for testing H against 0 > 0, or 
8 < 6,. Suppose that there exists a two-sided test pọ given by (3) of Chapter 
4, such that 


(36) sup [B3(8) = B,,(8)] = sup [B2(8) - &,,(0)]. 
peo 876; 


and that the supremum is attained on both sides, say at points 0, < bo < b>. 
If B, 0;) = Ba i = 1,2, an application of the fundamental lemma [Theorem 
5(iii) of Chapter 3] to the three points 6;, 0z, 6, shows that among all tests p 
with B,(0,) = Bi and B,(8,) = By, only qo satisfies B,(8)) < a. For any 
other level-a test, therefore, either 8,(0,) < B, or B, (8) < B,, and it 
follows that pọ is the unique most stringent test. The existence of a test 
satisfying (36) can be proved by a continuity consideration [with respect to 
variation of the constants C; and y; which define the boundary of the test 
(3) of Chapter 4] from the fact that for the UMP one-sided test against the 
alternatives 0 > 6, the right-hand side of (36) is zero and the left-hand side 
positive, while the situation is reversed for the other one-sided test. 


7. PROBLEMS 


Section 1 


tests. Let (£, #) be a Euclidean sample space, and let 
0 € Q, be dominated by a o-finite measure over (X, ). 
Qp of Q there exists a level-a test 


1. Existence of maximin 
the distributions Pj, 
For any mutually exclusive subsets Ry, 


Mn 2. 

[Let g= ae Ejo( X)), where the supremum 1s taken over all level-a Wes 

of H:0 € Qy Let P, be a sequence of level-a tests such ers M 3 
: gi teed by Theo: 

tends to B. If is a subsequence and 9 a test (guaran J 

i rises that Ej9,, (X) tends to Eyp(X) for all 0 € Q, then ọ is a 


level-a test and info, Eyp(X) = 81 
the distance of an 


tests. Let d be a measure of he d 
; Kee aa hypothesis H. A level-a test go is said to be nd 
Mess (LMP) if, given any other level-a test q, there exists A such that 


(37) RU 2 By(0) forall @ with 0< a(0) <A. 


Suppose that @ is real-valued and that the power func 
continuously differentiable at o- 


tion of every test is 
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(i) If there exists a unique level-a test pọ of H: 0 = maximizing £; (9), 
then pọ is the unique LMP levela test of H against @> 6, for 
d(0) — 0 — 6. 

(ii) To see that (i) is not correct without the uniqueness assumption, let X 
take on the values 0 and 1 with probabilities P,(0) = 4 — 0°, P,(1) = 
1 + 63, —1< 9? < 1, and consider testing H : 0 = 0 against K : 6 > 0. 
Then every test p of size a maximizes f; (0), but not every such test is 
LMP. [Kallenberg et al. (1984).] 

(iii) The following* is another counterexample to (i) without uniqueness, in 
which in fact no LMP test exists. Let X take on the values 0,1,2 with 
probabilities 


x 


P(x)=a+ {a + #?sin( 7 


J for x 1,2, 


P,(0) = 1— po(1) — po(2), 


where —1 < 0 < 1 and € is a sufficiently small number. Then a test q at 
level a maximizes 8'(0) provided 


9(1) + 9(2) 7 1; 


but no LMP test exists. 

(v) A unique LMP test maximizes the minimum power locally provided its 
power function is bounded away from a for every set of alternatives 
which is bounded away from H. 

(v) Let LTD X, be a sample from a Cauchy distribution with unknown 
location parameter 6, so that the joint density of the X's is 7 "I1/-,[1 
+ (x, — 8)^]-!. The LMP test for testing @ = 0 against @ > 0 at level 
a < } is not unbiased and hence does not maximize the minimum power 
locally. 


[(iii): The unique most powerful test against Ó is 
(1) ARS RS .[2 
(ori o (z) em). 


and each of these inequalities holds at values of @ arbitrarily close to 0. 

(Y): There exists M so large that any point with x, > M for all i=1,...,” 
lies in the acceptance region of the LMP test. Hence the power of the test tends 
to zero as @ tends to infinity.] 


A level-a test go is locally unbiased (loc. unb.) if there exists Ag > 0 such that 
B,,(8) = a for all à with 0 < d(@) < A,; it is LMP loc. unb. if it is loc. unb. 


*Due to John Pratt. 
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and if, given any other loc. unb. level-a test 9, there exists A such that (37) 
holds, Suppose that @ is real-valued and that d(@) = |@ — l, and that the 
power function of every test is twice continuously differentiable at 0 = 4. 


(G) If there exists a unique test po of H: 0 = 6 against K: 8 # 0 which 
among all loc. unb. tests maximizes B" (8), then gp is the unique LMP 
loc. unb. level-a test of H against K. 

Gi) The test of part (i) maximizes the minimum power locally provided its 
power function is bounded away from a for every set of alternatives that 
is bounded away from H. 


[(ii): A necessary condition for a test to be locally minimax is that it is loc. 


unb.] 
Section 2 
4. Let the distribution of X depend on the parameters (0,9) = 


(0,,.2:,0:,01, ten Og 9A test of H: 0 = 6? is locally strictly unbiased if for 
each 0, (a) B, (05, 8) = a, (b) there exists a @-neighborhood of 6° in which 


B, (0, 8) > a for 0 + 6°. 
(i) Suppose that the first and second derivatives 


ð ; CA 
&(9) = 3580.9 i and R/G) 7 3; 5 P 09 F 


d sufficient 


exist for all critical functions q and all 9. Then a necessary an 
0 for all i 


be locally strictly unbiased is that B0) = 
(Bi(9)) is positive definite for all 9. 


(ii) A test of H is said to be of 


the Gaussian curvature 
by (7) of Chapter 7 
Chapter 7 is of 


with this property m: 
à pus tions turns out to be equal to 


under the stated condi | 
face at 8°.) Then the test po given 


of the power sur! j ; 
a a to general linear univariate hypothesis (3) of 


type E. 
ii i = <.. 1 0), the test po, by Problem 
[(ii): With 0 = (ms) and 9 = Civ ede 
5 of Chapter 7, has the property of maximizing the surface integr: 


f[8. - e] a4 


tis provided by Cohen and Sackrowitz (1975), who 


i i D tes 
*An interesting example of a type- e 
show that the x?-test of Chapter 8, Example 5 has this property. 
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among all similar (and hence all locally unbiased) tests where S = 
((m;-.., 0): Eri? = plo? ). Letting p tend to zero and utilizing the condi- 
tions 


Bi(9) =0, [ma - 0 for i +j, f? 44 = Coo), 


one finds that pọ maximizes 17. 8, (v, a?) among all locally unbiased tests. 
Since for any positive definite matrix, (87 )| « I18;, it follows that for any 
locally strictly unbiased test p, 


- L Bi r X Ll r : 
Itn) « riae « [ER T [PA] - tns Iau) 

Let Z,,..., Z, be identically independently distributed according to a con- 
tinuous distribution D, of which it is assumed only that it is symmetric about 
some (unknown) point. For testing the hypothesis H : D(0) = }, the sign test 
maximizes the minimum power against the alternatives K: D(0) < q (q < 3). 
[A pair of least favorable distributions assign probability 1 respectively to the 
distributions F € H, G € K with densities 


VOR dE 


i r FEET mss 


7 


where for all x (positive, negative, or zero) [x] denotes the largest integer 
x] 


Let f(x) = Og(x) + (1 —6)h(x) with 0 «6 «1. Then fj(x) satisfies 
the assumptions of Lemma 1 provided g(x)/h(x) is a nondecreasing function 
of x. 


Let x =(x,,...,x,), and let gj(x,£) be a family of probability densities 
depending on 0 = (6,,..., 6.) and the real parameter £, and jointly measurable 
in x and £. For each 0, let hg(£) be a probability density with respect to a 
c-finite measure v such that pp(x) = fga(x, £)hg(£) d»(£) exists. We shall 
say that a function f of two arguments u = (u,..., u,), V = (01,5 0) 35 
nondecreasing in (u, v) if f(u’, v)/f(u, v) < f(u', v’)/f(u, v’) for all (u, v) 
satisfying u, Xu v SU (i=1,...,7; j =1,...,5). Then po(x) is nonde- 
creasing in (x, 0) provided the product g,(x, £)h (£) is (a) nondecreasing in 
(x, 8) for each fixed £; (b) nondecreasing in (0,£) for each fixed x; (O 
nondecreasing in (x, £) for each fixed 6. 

[Interpreting g,(x, £) as the conditional density of x given £, and A (£) as the 
a priori density of £, let p(£) denote the a posteriori density of £ given x, anı 
let p'(£) be defined analogously with 6’ in place of @. That pj(x) is nonde- 
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creasing in its two arguments is equivalent to 


ge(x^ £) 
ge (x, £) 


jm 


n i J 


p'(£) dv(£). 


By (a) it is enough to prove that 


gy (x^, £) 
S s: Loë) — o(€)] dv(£) > 0. 


Let S_ = (£: p/(£)/p(5) < 1) and S, = (£:p' 

5 : T : p(£)/p(£) = 1). By (b) the s 
S. S entirely to the left of S... It follows from (c) that FR des eu 
suci at 


D- af, Lo) — p(€)] dv(é) + of Lo) — p(£)] dv(£). 


and hence that D = (b — af LoC) — p(€)] dv(€) = 0] 


(i) Let X have binomial distribution b( p, n), and consider testing H : p = Po 
at level a against the alternatives 2x : p/q € 3po/4o % = 2 po/ qo. For 
a = .05 determine the smallest sample size for which there exists a test 
with power > .8 against Q if p 7 -1,.2, 3,4,.5. 

Gi) Let X,..., X, be independently distributed as N(£, o?). For testing 
o = Lat level a = .05, determine the smallest sample size for which there 

the alternatives o? < } and $22 


[See Problem 5 of Chapter 4.] 
Double-exponential distribution. 
double-exponential distribution wi 
testing 0 < 0 against 6 > 0 is the sign test, p! 
ISSNA 
a= x È (k) 


k=0 


Let X,..,X, be @ sample from the 
th density 1e". The LMP test for 
rovided the level is of the form 


so that the level-a sign test is nonrandomized. y 
set of the sample space 1n which k of the 


Let R, (k 2 0,.." be the sub: 

Ys Rx and pi k are negative. Let 0 < k<l<n, and let Sp, 5, be 
subsets of R4, Ry such that Po(S,) = P (Si) * 0- Then it follows from a 
consideration of Pa (Sk) and P,(S;) for small @ that there exists A such that 
P,(S,) < Po(S;) for 0 <6 <A. Suppose now that the rejection region of a 
nonrandomized test of 8 = 0 against 0 > 0 does not consist of the upper tail 


of a sign test. Then it can be converted into a sign test of the same size by a 
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finite number of steps, each of which consists in replacing an S, by an S, with 
k < l, and each of which therefore increases the power for 0 sufficiently small.] 

Section 3 
If (13) holds, show that q, defined by (11) belongs to 2. 


Show that there exists a unique constant b for which gq, defined by (11) is a 
probability density with respect to p, that the resulting gù belongs to 2), and 
that b > oo as €, > 0. 


Prove the formula (15). 
Show that if A + 2, and €g, e, are sufficiently small, then Q * Q). 


Evaluate the test (21) explicitly for the case that P, is the normal distribution 
with mean £, and known variance o?, and when «, = €. 


Determine whether (21) remains the maximin test if in the model (20) G, is 
replaced by G, . 


Write out a formal proof of the maximin property outlined in the last 
paragraph of Section 3. 


Section 4 
Let X,,..., X, be independently normally distributed with means E( X;) = p; 
and variance 1. The test of H: p = --- = p„ = 0 that maximizes the mini- 


mum power over w’: Yu, > d rejects when X X, > C. 

(if the least favorable distribution assigns probability 1 to a single point, 
invariance under permutations suggests that this point will be p, = ::* = bn 
= d/n]. 


(i) In the preceding problem determine the maximin test if w’ is replaced 
by Xa;p; > d, where the a’s are given positive constants. 
(i) Solve part (i) with Var( X,) = 1 replaced by Var(.X;) = o? (known). 


{@: Determine the point (pf... , u$) in w’ for which the MP test of H against 
K: (uf,---,4%) has the smallest power, and show that the MP test of H 
against K is a maximin solution.] 


Section 5 
Let X — (X,,..., X) and Y=(%,...,¥,) be independently distributed 


according to p-variate normal distributions with zero means and covariance 
matrices E(X,X;) = o,, and E(¥,Y,) = Ao. 


(i) The problem of testing H:A < A, remains invariant under the group G 
of transformations X* = XA, Y* = YA, where A = (a,,) is any nonsin- 
gular p X p matrix with a,,=0 for i» j, and there exists a UMP 
invariant test under G with rejection region Y2/X2 > C. 


*Due to Fritz Scholz. 
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(i) The test with rejection region Y?/X? > C maximizes the minimum 
power for testing A < A, against A> A, (A, < Aj). 
[(i): That the Hunt-Stein theorem is applicable to G can be proved 
in steps by considering the group G, of transformations X; = 
a, X, Tc +a,X,, X = X, fori-21..,q-lq*l...,p, sucess- 
sively for q = 1,..., p — 1. Here a, * 0, since the matrix A is nonsingu- 
lar if and only if a,, # 0 for all i. The group product (y,,..., y,) of two 
such transformations (a,,... , &4) and (f,..., 4) is given by y, = ah, 
+ Bi, n = 0B, + Ban: wa ,-1B, + By-1» Yq = a,B,, which 
shows G, to be isomorphic to a group of scale changes (multiplication of 
all components by B,) and translations [addition of (Bj... B, 1,0). The 
result now follows from the Hunt-Stein theorem and Example 7, since the 
assumptions of the Hunt-Stein theorem, except for the easily verifiable 
measurability conditions, concern only the abstract structure (G, @), and 
not the specific realization of the elements of G as transformations of 


some space.] 


Suppose that the problem of testing 0 € Qy against 0 € Xx remains invariant 
under G, that there exists a UMP almost invariant test pọ with respect to G, 
and that the assumptions of Theorem 3 hold. Then pọ maximizes 
infg, [w(8) Eg (X) + u(0)] for any weight functions w(0) > 0, u(0) that are 


invariant under G. 


Section 6 
Under the assumptions of Problem ] there 


Existence of most stringent tests. 
: against 0 € Q — Qy. 


exists a most stringent test for testing 0e€Q, 
sets of alternatives such that the 
and that UQ, = 2 — Ny, 
= ¢ is independent of 


Let (Q4) be a class of mutually exclusive 
envelope power function is constant over each Q1 
and let q, maximize the minimum power over QA. If Pa 
A, then g is most stringent for testing 0 € €. 
distributed according to the 
Let (Z,,..., Zy) = (hos way E) be ording to U 
hxc (56) of Chapter 5, and consider the problem of ae Hin ae 
against the alternatives oan i Ks dependently normally 
distributed with common variance 9 an means 7 * §- 
test with rejection region |¥ -X|> C[T(Z)), the two-si 
f Chapter 5, is most stringent. 4 . 
We Problem 22 with each of the sets 0, consisting of two points (5i, m. 9 ), 


(£5, m, 0) such that 


ded version of the test 


m 
g = —— ô; 
eet ™ beet 


m 
ô 


" = 
fy hae nos m+n 


for some f and ô.] 
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Additional Problems 
Let X,,..., X, be independent normal variables with variance 1 and means 
&,,..., n, and consider the problem of testing H: $, = --- = £, = 0 against 


the alternatives K = {K,,..., K,), where K; : £; = 0 for j # i, $; = $ (known 
and positive). Show that the problem remains invariant under permutation of 
the X's and that there exists a UMP invariant test ¢) which rejects when 
Xe-55 > C, by the following two methods. 


(i) The order statistics Xa) < +- < X,,, constitute a maximal invariant. 
(ii) Let fọ and f, denote the densities under H and K, respectively. Then the 


level-a test ¢ of H vs. K': f = (1/n)E f, is UMP invariant for testing 
H vs. K. 


[(ii): If $5 is not UMP invariant for H vs. K, there exists an invariant test $, 
whose (constant) power against K exceeds that of $). Then $, is also more 
powerful against K’.] 


The UMP invariant test ¢) of Problem 24 
(i) maximizes the minimum power over K; 
(i) is admissible. 
(iii) For testing the hypothesis H of Problem 24 against the alternatives 
K' = (Ky,..., Kns Kj,..., Ki}, where under K/: €, = 0 for all j+ i, 


$; = —£, determine the UMP test under a suitable group G’, and show 
that it is both maximin and invariant. 


[ii): Suppose $' is uniformly at least as powerful as $), and more powerful for 

at least one K,, and let 

E9(x,,....x,) 
n! 


where the summation extends over all permutations. Then ó* is invariant, and 
its power is independent of i and exceeds that of $,.] 


Qt(xiyem xn) T 


, 


Show that the UMP invariant test of Problem 24 is most stringent. 


For testing H: f, against K: {fi,---,f,}, suppose there exists a finite group 
G = {g),..., gy) which leaves H and K invariant and which is transitive in 
the sense that given f., f. (1 < j, j’) there exists g € G such that gf, = fj. In 
generalization of Problems 24, 25, determine a UMP invariant test, and show 
that it is both maximin against K and admissible. 


To generalize the results of the preceding problem to the testing of H: f. vs- 
K: { fo, 9 € o), assume: 


(i) There exists a group G that leaves H and K invariant. 
(i) G is transitive over w. 
(iii) There exists a probability distribution Q over G which is right-invariant 
in the sense of Section 4. 


i REFERENCES tee 


Determine a UMP invariant test, and show that it is both maximin against K 
and admissible. 


29. Let X,,..., X, be independent normal with means 6,,..., 0, and variance 1. 


(i) Apply the results of the preceding problem to the testing of H : 0, = --- 
= 6, = 0 against K:X0? = r°, for any fixed r > 0. 
(ii) Show that the results of (i) remain valid if H and K are replaced by 
H' : X0} < r, K’:26? > rd (ry € n). 
30. Suppose in Problem 29(i) the variance 6? is unknown and that the data consist 
of X,,...,X, together with an independent random variable S? for which 
S?/o* has a x?-distribution. If K is replaced by X602 /o? = r°, then 


(i) the confidence sets E(6, — X;)?/S* < C are uniformly most accurate 
equivariant under the group generated by the n-dimensional generaliza- 
tion of the group Gp of Example 17 of Chapter 6, and the scale changes 
X! = cX, S? = oS. 

(i) The confidence sets of (i) are minimax with respect to the measure p 
given by 


nl c(X, 8?)] = [volume of c(x, S?)). 


[Use polar coordinates with 0? = x02] 
ij is finite and 
31. Locally uniformly most powerful tests. If the sample space is 
independent of 6, the test Po of Problem 2(i) is not only LMP but also locally 
uniformly most powerful (LUMP) in the sense that there exists a value A > 0 
such that pọ maximizes f, (6) for all 6 with 0 < Us b <A. 
[See the argument following (19) of Chapter 6, Section 9.] 
32. The following two examples show that the assumption of a finite sample space 
is needed in Problem 31. 


() Let X... X, be iid. according to a normal distribution N(o, o^) and 


test H: o = ay against K: 0 > 0- à; 

(ii) Let X and Y be independent Poisson variables with E(X) = à me 

E(Y) =A +1, and test H:A=Ao against t In eacl ` 
determine the LMP test and show that it is not LUMP. 

a simple 


[Compare the LMP test with the most powerful test against 


alternative.] 
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C H A PE Ra 


Conditional Inference 


1. MIXTURES OF EXPERIMENTS 


The present chapter has a somewhat different character from the preceding 
ones. It is concerned with problems regarding the proper choice and 
interpretation of tests and confidence procedures, problems which—despite 
a large literature—have not found a definitive solution. The discussion will 
thus be more tentative than in earlier chapters, and will focus on conceptual 
aspects more than on technical ones. 

Consider the situation in which either the experiment & of observing a 
random quantity X with density p, (with respect to p) or the experiment F 
of observing an X with density qj (with respect to y) is performed with 
probability p and q = 1 — p respectively. On the basis of X, and knowl- 
edge of which of the two experiments was performed, it is desired to test 
Hy: = b, against H,:4= 9). For the sake of convenience it will be 
assumed that the two experiments have the same sample space and the same 
o-field of measurable sets. The sample space of the overall experiment 


consists of the union of the sets 


X= ((I,x):1 20 xe£) and q,- ((I,.x):1=1,%€ 2) 


where J is 0 or 1 as & or F is performed. ! : 
A level-a test of Hp is defined by its critical function 


g(x) =¢li, x) 
and must satisfy 
(1) pE sfo ( Xle] + PACIA = p [oe du S ao, dv <a. 


Suppose that p is unknown, so that Ho is composite. Then a level-a test of 
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H, satisfies (1) for all 0 « p « 1, and must therefore satisfy 


Q) ao = f dopo, du < a and a = forgo, dv < a. 


As a result, a UMP test against H, exists and is given by 


(3) &G) = : it S Bes (x) = n if an Be, 
where the c, and y, are determined by 
(4) E, |e( X)|6] = E, [e.( X)| 7] = a. 
The power of this test against H, is 
(5) B(p) = PB, + 4B, 
with 
(6) Bo = En [XE], By = E [e O01 8]. 


The situation is analogous to that of Chapter 4, Section 4, and, as was 
discussed there, it may be more appropriate to consider the conditional 
power B, when J = i, since this is the power pertaining to the experiment 
that has been performed. As in the earlier case, the conditional power f; 


can also be interpreted as an estimate of the unknown B(p), which is 
unbiased, since 


E(B;) = pBy + qb, = B(p). 


So far, the probability p of performing experiment & has been assumed 
to be unknown. Suppose instead that the value of p is known, say p = i 
The hypothesis H can be tested at level « by means of (3) as before, but the 
power of the test js now known to be 1(B, + B,). Suppose that By = 3 
B, = 9, so that at the start of the experiment the power is 1(.3 + .9) = 6. 
Now a fair coin is tossed to decide whether to perform & (in case of heads) 
or F (in case of tails). If the coin shows heads, should the power be 
reassessed and scaled down to .32 

Let us postpone the answer and first consider another change resulting 
from the knowledge of p. A level-a test of H now no longer needs to satisfy 
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(2) but only the weaker condition 


1 
(7) a [ons a + fort, >| <a. 


The most powerful test against K is then again given by (3), but now with 
Co = ¢, = ¢ and ~=% Y determined by (Problem 3) 


(8) (a + a) =a, 
where 
(9) ao = E Olé]. a = Enl]. 


As an illustration of the change, suppose that experiment is reason- 
ably informative, say that the power B, given by (6), is .8, but that & has 
little ability to distinguish between Po, and pj, Then it will typically not 
pay to put much of the rejection probability into a; if By [given by (6)] is 
sufficiently small, the best choice of a and a, satisfying (8) is approxi- 
mately a = 0, a, = 2a. The situation will be reversed if 7 is so informa- 
tive that F can attain power close to 1 with an a, much smaller than a/2. 

When p is known, there are therefore two issues. Should the procedure 


be chosen which is best on the average over both experiments, or should the 
best conditional procedure be preferred; and, for a given test or confidence 
as level, power, and confidence coeffi- 


procedure, should probabilities such 
cient be calculated conditionally, given the experiment that has been selected, 
or unconditionally? The underlying question is of course the same: Is à 
conditional or unconditional point of view more appropriate? 

The answer cannot be found within the model but depends on the 


i i times, for 

context. If the overall experiment will be performed many 
i i jal or agricultural setting, the average performance 
example in an industrial o gri pei ere PE iam 


may be the principal feature of interest, and i 

shitablel ee repetitions refer to different clients, or dir 
rather than actual, interest will focus on the particular bp € 3 m 
conditioning seems more appropriate. Unfortunately, as peer 
sections, it is then often not clear how the conditioning even! 


: T 

The RED between the conditional and the unconditional aped 
tends to be most striking, and a choice between ee Lanne 
pressing, when the two experiments 4 and F differ sins ; reor 
of information they contain, if for example the Miami A a win is 
large. To illustrate an extreme situation 1n whic! s 
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suppose that & and F consist in observing X with distribution N(@,1) and 
N( — 0, 1) respectively, that one of them is selected with known probabilities 
p and q respectively, and that it is desired to test H: 0 — 0 against 
K:0 > 0. Here & and F contain exactly the same amount of information 
about 6. The unconditional most powerful level-a test of H against 0, > 0 
is seen to reject (Problem 5) when X » c if & is performed, and when 
X < —cif F is performed, where P(X > c) = a. The test is UMP against 
0 > 0, and happens to coincide with the UMP conditional test. 

The issues raised here extend in an obvious way to mixtures of more than 
two experiments. As an illustration of a mixture over a continuum, consider 


a regression situation. Suppose that X,,..., X, are independent, and that 
the conditional density of X; given 1, is 

lal x= a= t, 

rma) 

o c 


The 1, themselves are obtained with error. They may for example be 
independently normally distributed with mean c, and known variance 7°, 
where the c; are the intended values of the ;;. Then it will again often be the 
case that the most appropriate inference concerning a, f, and c is condi- 
tional on the observed values of the ’s (which represent the experiment 
actually being performed). Whether this is the case will, as before, depend 
on the context. 

The argument for conditioning also applies when the probabilities of 
performing the various experiments are unknown, say depend on a parame- 
ter 2, provided $ is unrelated to 6, so that which experiment is chosen 
provides no information concerning 0. A more precise statement of this 
generalization is given at the end of the next section. 


2. ANCILLARY STATISTICS 


Mixture models can be described in the following general terms. Let (6; 
Z € Z] denote a collection of experiments of which one is selected accord- 
ing to a known probability distribution over 2. For any given z, the 
experiment 4; consists in observing a random quantity X, which has a 
distribution Pe(-|z). Although this structure seems rather special, it is 
common to many statistical models. 

Consider a general statistical model in which the observations X are 
distributed according to Pj, 0 € Q, and suppose there exists an ancillary 
statistic, that is, a statistic Z whose distribution F does not depend on 9. 
Then one can think of X as being obtained by a two-stage experiment: 
Observe first a random quantity Z with distribution F; given Z = z, 
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observe a quantity X with distribution P,(-|z). The resulting X is distrib- 
uted according to the original distribution Pj. Under these circumstances, 
the argument of the preceding section suggests that it will frequently be 
appropriate to take the conditional point of view.* (Unless Z is discrete, 
these definitions involve technical difficulties concerning sets of measure 
zero and the existence of conditional distributions, which we shall disregard.) 

An important class of models in which ancillary statistics exist is ob- 
tained by invariance considerations. Suppose the model 2 = { Py, 0 € Q} 
remains invariant under the transformations 


XogX, 0585 g eG, geG, 


and that G is transitive over Q.* 


Theorem 1. Jf P remains invariant under G and if G is transitive over Q, 
then a maximal invariant T (and hence any invariant) is ancillary. 


Proof. It follows from Theorem 3 of Chapter 6 that the distribution ofa 
maximal invariant under G is invariant under G. Since G is transitive, only 
constants are invariant under G. The probability P,(T € B) is therefore 
constant, independent of 8, for all B, as was to be proved. 

As an example, suppose that X = (5,..., Xn) is distributed po 
to a location family with joint density f(x = 6. X7 6). The mos 
powerful test of H: 0 = 6, against K : 0 = 0, > 6, rejects when 


f(x eT E EA, 
JS T e 


(10) Fx apte E 


X, X, (i7 dyes 7 1) is ancillary. 
i 


This is obvious by inspection and follows from Theorem 1 r eo 
with Example 1(i) of Chapter 6. It may therefore be more pp 


iti i -5 ed aston diet 
consider the testing problem conditionally given Los wi d ie mn 
To determine the most powerful conditional test, tr A 
where Y, — X,. The conditional density of Y, given Yy.» Jn-1 


Here the set of differences Y, — 


5th In 9 


» fO t» - 9 
(1). poi 7 ffo eet u) du 


ituati lying on aspects 
i i and the present situation, rel 
+A distinction between experimental mixtures nates 
NER is discussed by Basu (1964) an yer settee 4 
+The family 2 is then a group family; see TPE, Chap 
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and the most powerful conditional test rejects when 


Pa lo 823) 


12 SEC Vi) 
qa Po lis 98i) i : 


In terms of the original variables this becomes 


y Area nece e Cr) 


un f(x) — 05,..., x, — 8) 


-c(x;—2x,...,x,:4 7 x,). 


The constant c(x, — x,,..., x, , — x,) is determined by the fact that the 
conditional probability of (13), given the differences of the x's, is equal to a 
when 6 = bọ. 

For describing the conditional test (12) and calculating the critical value 
C(Yi»---» Yn—1)) it is useful to note that the statistic Y, = X, could be 
replaced by any other Y, satisfying the equivariance condition* 


(14) Y,(x, +.4,...,x,+4) =Y,(x,...,x,)+@ foral a. 


This condition is satisfied for example by the mean of the X's, the median, 
or any of the order statistics. As will be shown in the following Lemma 1, 
any two statistics Y, and Y/ satisfying (14) differ only by a function of the 
differences Y, = X, — X, (i = 1,...,n — 1). Thus conditionally, given the 
values y,,..., y, , Y, and Y; differ only by a constant, and their condi- 
tional distributions (and the critical values c(y,,..., y, .,) differ by the 
same constant. One can therefore choose Y,, subject to (14), to make the 
conditional calculations as convenient as possible. 


Lemma 1. Jf Y, and Y; both satisfy (14), then their difference ^ = Y; — 
Y, depends on (x,,..., x,) only through the differences (xy — Xp» ---» Xn-1 
— XM 


Proof. Since Y, and Y/ satisfy (14), 
A(x, + a,...,x, +a) =A(x,,...,x,) for all a. 
Putting a = —x,, one finds 
A(x,,..., Xp) = A(x, — x,,... Xp_1 — x0). 
which is a function of the differences. 


*For a more detailed discussion of equivariance, see TPE, Chapter 3. 
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The existence of ancillary statistics is not confined to models that remain 
invariant under a transitive group G. The mixture and regression examples 
of Section 1 provide illustrations of ancillaries without the benefit of 
invariance. Further examples are given in Problems 8-13. 

If conditioning on an ancillary statistic is considered appropriate because 
it makes the inference more relevant to the situation at hand, it is desirable 
to carry the process as far as possible and hence to condition on a maximal 
ancillary. An ancillary Z is said to be maximal if there does not exist an 
ancillary U such that Z = f(U) without Z and U being equivalent. [For a 
more detailed treatment, which takes account of the possibility of modifying 
statistics on sets of measure zero without changing their probabilistic 
properties, see Basu (1959).] 

Conditioning, like sufficiency and invariance, leads to a reduction of the 
data. In the conditional model, the ancillary is no longer part of the random 
data but has become a constant. As a result, conditioning often leads to a 
great simplification of the inference. Choosing a maximal ancillary for 
conditioning thus has the additional advantage of providing the greatest 


reduction of the data. i 
Unfortunately, maximal ancillaries are not always unique, and one must 
for conditioning. [This 


then decide which maximal ancillary to choose l 
problem is discussed by Cox (1971) and Becker and Gordon (1983)] I 
invariant under a given 


attention is restricted to ancillary statistics that are i i ud 
group G, the maximal ancillary of course coincides with the maxim 


invariant. | i 

Another issue concerns the order in which to apply reduction by 
sufficiency and ancillarity. 

xamp. i i distributed according to 

E: le 1. Let X, Y) i = 1, -, m, be independently z 4 

a bivariate normal Oana with E(X) = EY) = vee Var(Y,) = 1 


uted as N(0,1) and are therefore ancillary. The coi 
X, = xisco Xa = Xn ÍS 


1 2 
Cexp = (Teak - px) ) 


"nen 2 E 
with the sufficient statisties Y Ping that (Y... 1) ie andan The 


ively, one co i d is Zy, then admits the 
conditional distribution of the X’s given hh = Ji atay V does not exist in 
i istics (EXE x. A unique maii Se functions of V. Thus 
sufficient statistics ( p ‘round Y's would have to be functions as M 
this case, NN to the full sample (Xi Y o (Xn Y) 
ancillary. 
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Suppose instead that the data are first reduced to the sufficient statistics T — 
(EX? + YY2,E X,Y,). Based on T, no nonconstant ancillaries appear to exist.* This 
example and others like it suggest that it is desirable to reduce the data as far as 
possible through sufficiency, before attempting further reduction by means of 
ancillary statistics. 


Note that contrary to this suggestion, in the location example at the 
beginning of the section, the problem was not first reduced to the sufficient 
statistics Xa) < --- < X). The omission can be justified in hindsight by 
the fact that the optimal conditional tests are the same whether or not the 
observations are first reduced to the order statistics. 

In the structure described at the beginning of the section, the variable Z 
that labels the experiment was assumed to have a known distribution. The 
argument for conditioning on the observed value of Z does not depend on 
this assumption. It applies also when the distribution of Z depends on an 
unknown parameter 3, which is independent of 0 and hence by itself 
contains no information about 6, that is, when the distribution of Z 
depends only on 2, the conditional distribution of X given Z = z depends 
only on 6, and the parameter space Q for (0, 0) is a Cartesian product 
Q = Q X Qs, with 


(15) (0,9)eQ = 6EQ, and 9€Q,. 


(the parameters 9 and $ are then said to be variation-independent, or 
unrelated.) 

Statistics Z satisfying this more general definition are called partial 
ancillary or S-ancillary. (The term ancillary without modification will be 
reserved here for a statistic that has a known distribution.) Note that if 
X — (T, Z) and Z is a partial ancillary, then T is a partial sufficient statistic 
in the sense of Chapter 3, Problem 36. For a more detailed discussion of this 
and related concepts of partial ancillarity, see for example Basu (1978) and 
Barndorff-Nielsen (1978). 


Example 2. Let X and Y be independent with Poisson distributions P(A) and 
P(p), and let the parameter of interest be @ = p/À. It was seen in Chapter 4, 
Section 4 that the conditional distribution of Y given Z = X + Y =z is binomial 
b(p, z) with p = p/(À + p) = 0/(8 + 1) and therefore depends only on 6, while 
the distribution of Z is Poisson with mean 9 — A + y. Since the parameter space 
0 < À, p < oo is equivalent to the Cartesian product of 0 < 0 < oo, 0 < ẹ < œ, it 
follows that Z is S-ancillary for 0. 

The UMP unbiased level-a test of H : p < À against p > A is UMP also among 
all tests whose conditional level given z is a for all z. (The class of conditional tests 
coincides exactly with the class of all tests that are similar on the boundary p = A.) 


"So far, nonexistence has not been proved. It seems likely that a proof can be obtained by 
the methods of Unni (1978). 
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When Z is S-ancillary for 6 in the presence of a nuisance parameter #, 
the unconditional power B(@, 9) of a test p of H : 0 = 6, may depend on 9 
as well as on 6. The conditional power B(9|z) = Eg[q( X)|z] can then be 
viewed as an unbiased estimator of the (unknown) f(0, à), as was discussed 
at the end of Chapter 4, Section 4. On the other hand, if no nuisance 
parameters ® are present and Z is ancillary for 6, the unconditional power 
B(8) = Egq( X) and the conditional power f(6|z) provide two alternative 
evaluations of the power of p against 6, which refer to different sampling 
frameworks, and of which the latter of course becomes available only after 
the data have been obtained. 

Surprisingly, the S-ancillarity of X + Y in Example 2 does not extend to 
the corresponding binomial problem. 


Example 3. Let X and Y have independent binomial distributions b(p,, m) 
and b(p;,nm) respectively. Then it was seen in Chapter 4, Section 5 that the 
conditional distribution of Y given Ze X * Y —z depends only on the cross- 
product ratio A = p;q,/pid; (qi = 1 — pi); However, Z is not S-ancillary for A. 
To see this, note that S-ancillarity of Z implies the existence of a parameter à 
unrelated to A and such that the distribution of Z depends only on $. As A 
changes, the family of distributions (5, 9€ $,) of Z would remain unchanged. 
This is not the case, since Z is binomial when A = 1 and not otherwise (Problem 
15). Thus Z is not S-ancillary. 

In this example, all unbiased sts 
that is independent of z, but conditio: 


ts of H: A = A, have a conditional level given z 
adi ning on Z cannot be justified by S-ancillarity. 


situation of the multinomial 2x2 


this example is the 
eee á of unbiasedness in Chapter 4, Sec- 


table discussed from the point of view 


tion 6. 
Example 4. In the notation of Cur ^ E $1 i id ps js apr Tat 4 
2 x 2 table be pu Pd wit total sample size M+N=T+T "=s. 


totals X+ Ye T, X'+ Y =T, 3 i = (b, h) = 
Hes it is easy to check thar, Uu URS S-ancillary for 0 = (fi, 6.) 
(p, , Dap/Da) with 9 = pp. Since the cross prone z 
a EC n 6,), it may be appropriate eget ia ooh = 
(M, N). Exactly analogously me. bos Oe un LU if quy be eal 
in/Pi)y and inte of ight t the set o 
ASA a test D H on (P T E A me ; ipid 
fou inals (M, N, T, T) = Z wo g 
trom pu example that this is not the c 
Here, all unbiased tests 
S-ancillarity permits con 
guidance as to which of the tw 


aditional level given z. However, 
(without giving any 


i d con- 
:nciple of carrying out tests an 
im Ss A or S-ancillaries frequently 


Despite such difficulties, 
to the corresponding unconditional proce- 


fidence estimation conditiona 
provides an attractive alternative 
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dures, primarily because it is more appropriate for the situation at hand. 
However, insistence on such conditioning leads to another difficulty, which 
is illustrated by the following example. 


Example 5. Consider N populations II,, and suppose that an observation X, 
from II, has a normal distribution N(£,,1). The hypothesis to be tested is 
Hif = +++ = £y. Unfortunately, N is so large that it is not practicable to take an 
observation from each of the populations; the total sample size is restricted to be 
n « N. A sample II,,...,IIj of n of the N populations is therefore selected at 
random, with probability 1/| X; ) for each set of n, and an observation X, is 
obtained from each of the populations II, in the sample. 

Here the variables J,,..., J, are ancillary, and the requirement of conditioning 
on ancillaries would restrict any inference to the n populations from which 
observations are taken. Systematic adherence to this requirement would therefore 
make it impossible to test the original hypothesis H.* Of course, rejection of the 
partial hypothesis Hy: = <--> = £j, would imply rejection of the original 
H. However, acceptance of H, ....;, would permit no inference concerning H. 

The requirement to condition in this case runs counter to the belief that a sample 
may permit inferences concerning the whole set of populations, which underlies 
much of statistical practice. 


With an unconditional approach such an inference is provided by the test with 
rejection region 


sf (Sf 


where c is the upper a-percentage point of x? with n — 1 degrees of freedom. Not 
only does this test actually have unconditional level a, but its conditional level given 
J = Jy... Ja = j, also equals a for all (j,,..., j,). There is in fact no difference in 
the present case between the conditional and the unconditional test: they will accept 
or reject for the same sample points. However, as has been pointed out, there is a 
AM difference between the conditional and unconditional interpretations of the 
results. 

If i EE P £,) denotes the conditional power of this test given 
I, =j: -+ J, = ja, its unconditional power is 


summed over all (3) n-tuples jy < --- <j,. As in the case with any test, the 


conditional power given an ancillary (in the present case J,,..., J,) can be viewed 
as an unbiased estimate of the unconditional power. 


*For other implications of this requirement, called the weak conditionality principle, see 
Birnbaum (1962) and Berger and Wolpert (1984). 
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3. OPTIMAL CONDITIONAL TESTS 


Although conditional tests are often sensible and are beginning to be 
employed in practice [see for example Lawless (1972, 1973, 1978) and 
Kappenman (1975)], not much theory has been developed for the resulting 
conditional models. Since the conditional model tends to be simpler than 
the original unconditional one, the conditional point of view will frequently 
bring about a simplification of the theory. This possibility will be illustrated 
in the present section on some simple examples. 


Example 6. Specializing the example discussed at the beginning of Section 1, 
suppose that a random variable is distributed according to N(0, o7) or N(0, aĝ) as 
I — 1 or 0, and that P(T = 1) = P(I = 0) =}. Then the most powerful test of 
H: 0 = 6, against 0 = 0, (> 4) based on (J, X) rejects when 


x- 3(% + %) 
SS ae 
207 5 


A UMP test against the alternatives 0 > 6; therefore does not exist. On the other 
hand, if H is tested conditionally given I = i, a UMP conditional test exists and 
rejects when X > c, where P(X > c |I = i) = a for i = 0,1. 


The nonexistence of UMP unconditional tests found in this example is 


typical for mixtures with known probabilities of two or more families with 


monotone likelihood ratio, despite the existence of UMP conditional tests in 


these cases. 
istribution N(£, a^^), 
L Lets be a sample from a normal distribution 
g > Oe pes coefficient of variation a > 0, a ce he problem ig 
ing H: = > against K: È> d. Here T= (T, %) with Ti e X, T 
M cr and Z = L/T, is ancillary. If we let V = yn T;/a, the 


= /(1/n)E X? is sufficient, I 
i density of V given Z =z is equal to (Problem 18) 


pel] 
(16) poe) = dem -5 ron WENN? 


The density has monotone likelihood ratio, so that the rejection region V > C(z) 
constitutes a OMM =L(X%- X)? are independent with joint den- 
nco; 5 


sity 
1 
n ni x EGET ! 
a» sem - xe) asp 
iscussi i le, see Hinkley 
and a UMP test does not exist. [For further discussion of this example 


(1977).] 
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An important class of examples is obtained from situations in which the 
model remains invariant under a group of transformations that is transitive 
over the parameter space, that is, when the given class of distributions 
constitutes a group family. The maximal invariant V then provides a natural 
ancillary on which to condition, and an optimal conditional test may exist 
even when such a test does not exist unconditionally. Perhaps the simplest 
class of examples of this kind are provided by location families under the 
conditions of the following lemma. 


Lemma 2. Let X,..., X, be independently distributed according to 
f(x, — 9), with f strongly unimodal. Then the family of conditional densities of 
Y, = X, given Y, — X, - X, (i- l...,n— 1) has monotone likelihood 
ratio. 


Proof. The conditional density (11) is proportional to 
(18) ft» 79): ft» - 9)fG, - 9). 


By taking logarithms and using the fact that each factor is strongly 
unimodal, it is seen that the product is also strongly unimodal, and the 
result follows from Example 1 of Chapter 9. 

Lemma 2 shows that for strongly unimodal f there exists a UMP 
conditional test of H : 0 < 0, against K : @ > 6), which rejects when 


(19) pono es o Ya — X:). 


Conditioning has reduced the model to a location family with sample size 
one. The double-exponential and logistic distributions are both strongly 
unimodal (Section 9.2), and thus provide examples of UMP conditional 
tests. In neither case does there exist a UMP unconditional test unless 
n=1. 

As a last class of examples, we shall consider a situation with a nuisance 
parameter. Let X,,..., Xm and Y,,..., Y, be independent samples from 
location families with densities f(x, — ,..., Xm — £) and g(yi — I++? 
Ya — N) respectively, and consider the problem of testing H : v < £ against 
K: > £. Here the differences U, = X, — X,, and V, = Y, — Y, are ancillary. 
The conditional density of X = X,, and Y = Y, given the u’s and v’s is 
seen from (18) to be of the form 


(20) Occ E)gsy- 7); 


where the subscripts u and v indicate that f* and g* depend on the u's and 
v's respectively. The problem of testing H in the conditional model remains 
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invariant under the transformations: x’ = x + c, y' =y * c, for which 
Y — X is maximal invariant. A UMP invariant conditional test will then 
exist provided the distribution of Z = Y — X, which depends only on 
A = ņ — £, has monotone likelihood ratio. The following lemma shows that 
a sufficient condition for this to be the case is that f* and g* have 
monotone likelihood ratio in x and y respectively. 

Lemma 3. Let X, Y be independently distributed with densities f*(x — £), 
g*(y — n) respectively. If f* and g* have monotone likelihood with respect to 
¢ and n, then the family of densities of Z = Y — X has monotone likelihood 
ratio with respect to A= — $. 


Proof. The density of Z is 
(21) hy) = fetG - A)f*(y - 2). 


To see that A,(z) has monotone likelihood ratio, one must show that for 
any A < A’, hy(z)/hg(z) is an increasing function of z. For this purpose, 
write 

hu) yeu -A) gato e. 

ha(z) g*(y- 4) feu- Aflu- z) du 


The second factor is a probability density for Y, 
(22) p.) = Gg" -Af - 2» 


which has monotone likelihood ratio in the parameter z b 
made about f*. The ratio 


y the assumption 


hale) tg) 
(23) Pm leew phe) Y 


is the expectation of ET > pA rocca WE EN ME sa 
ton 1 pe a auta 2 of Chapter 3 that its expectation 1s 
“awe om, Os E Tp gh 
ae ui pieces unimodal. sss mr ARPA 
conditional distribution ha(2) then has monotone 
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3, and a UMP conditional test exists and rejects for large values of Z. (This 
result also follows from Problem 7 of Chapter 9). 

The difference between conditional tests of the kind considered in this 
section and the corresponding (e.g., locally most powerful) unconditional 
tests typically disappears as the sample size(s) tend(s) to infinity. Some 
results in this direction are given by Liang (1984); see also Barndorff- 
Nielsen (1983). 

The following multivariate example provides one more illustration of a 
UMP conditional test when unconditionally no UMP test exists. The results 
will only be sketched. The details of this and related problems can be found 
in the original literature reviewed by Marden and Perlman (1980) and 
Marden (1983). 


Example & The normal multivariate two-sample problem with covariates was 


seen in Chapter 8, Example 3, to reduce to the canonical form (the notation has 
been changed) of m + 1 independent normal vectors of dimension p = p, + pz, 


Verr T SN mz eo Zar 
with common covariance matrix X and expectations 

E(Y)-», E(YX)-E(Z)- -` = E(Z,,) = 0. 
The hypothesis being tested is H: y, = 0. Without the restriction E(Y;) = 0, the 
model would remain invariant under the group G, of transformations (Chapter 8, 
Section 2): Y* = YB, Z* = ZB, where B is any nonsingular p X p matrix. How- 


ever, the stated problem remains invariant only under the subgroup G' in which B 
is of the form [Problem 22(i)] 
rie ( By, 0 L^ 


By By P2 d 


Pi P2 
If 


Su S z z 
ZZ=S= 11 12 d = 11 12 
( Sy d " J Zn Zn i 


the maximal invariants under G’ are the two statistics D = Y, S}? Y/ and 


1+D i 


and the joint distribution of (N, D) depends only on the maximal invariant 


ig RELEVANT SUBSETS 553 
under G’, 


A= m(Zu = Sy un. 


The statistic D is ancillary [Problem 22(ii)], and the conditional distribution of N 
given D = d is that of the ratio of two independent x?-variables: the numerator 
noncentral x? with p degrees of freedom and noncentrality parameter A/(1 + d), 
and the denominator central x? with m -- 1 — p degrees of freedom. It follows 
from Chapter 7, Section 1, that the conditional density has monotone likelihood 
ratio. A conditionally UMP invariant test therefore exists, and rejects H when 
(m + 1 — p) N/p > C, where C is the critical value of the F-distribution with p 
and m + 1 — p degrees of freedom. On the other hand, a UMP invariant (uncondi- 
tional) test does not exist; comparisons of the optimal conditional test with various 
competitors are provided by Marden and Perlman (1980). 


4, RELEVANT SUBSETS 


The conditioning variables considered so far have been ancillary statistics, 
i.e. random variables whose distribution is fixed, independent of the param- 
eters governing the distribution of X, or at least of the parameter of interest. 
We shall now examine briefly some implications of conditioning without 
this constraint. Throughout most of the section we shall be concerned with 
the simple case in which the conditioning variable is the indicator of some 
subset C of the sample space, so that there are only two conditioning events 
1-1 (ie. XE cC) and I 7 0 (ie. XecC, the complement of C). es 
mixture problem at the beginning of Section 1, with 2, = C and 2) = C, is 
of this type. 

Suppose X is distributed with di i 
region for testing the simple hypothesis 
alternatives. For any subset C of the samp! 
rejection probabilities 


lensity pp, and R is a level-a rejection 
H: 6 = 0 against some class of 
le space, consider the conditional 


(24) ac = Py(XERIC) and ac= A(X€ RIC), 


i ifficult 
and suppose that ap» & and ac <4. Then we are in the difficulty 


i , the probability of falsely 
describe ia PA yu om have fallen into C 


rejecti was stated to be a. Now : i 
(or OE the original statement be adjusted sd; pies rude 
(or lower value ag) be quoted? An extreme ae s Leap 
when C is a subset of R or R, since then Ki i AT 
It is clearly always possible to chose C so aA ae die popes 
exceeds the stated a. It is not so clear vee aie 
always exists for the levels of a family of co 
inequality must now hold for all 0. 


554 CONDITIONAL INFERENCE [10.4 


Definition. A subset C of the sample space is said to be a negatively 
biased relevant subset for a family of confidence sets S( X) with uncondi- 
tional confidence level y = 1 — a if for some € > 0 
(25) ¥c(0) = P [0E S(X)|XEC] <y-« forall, 
and a positively biased relevant subset if 


(26) Pj8&es(x)xec|»v-e forall @. 


The set C is semirelevant, negatively or positively biased, if respectively 


(27) P,[8ES(X)|XEC] <y forall @ 
or 
(28) PBjees(x)xec]|zv  foral6, 


with strict inequality holding for at least some 6. 


Obvious examples of relevant subsets are provided by the subsets Z 
and F, of the two-experiment example of Section 1. 

Relevant subsets do not always exist. The following four examples 
illustrate the various possibilities. 


Example 9. Let X be distributed as N(0,1), and consider the standard con- 
fidence intervals for 0: 


S(X) = {0:X-c<0<X+c}, 


where ©(c) — ®(—c) = y. In this case, there exists not even a semirelevant subset. 


2 P see this, suppose first that a positively biased semirelevant subset C exists, So 
a 


A(8) = R[X-c<0< X+ cand Xe C] —yR[XEC]>0 


for all 0, with strict inequality for some 6. Consider a prior normal density (9) 
for @ with mean 0 and variance 72, and let 


B(x) = P[x -c< 8 <x + ex], 


where 8 has density (9). The posterior distribution of € given x is then normal 
with mean 77x/(1 + 7?) and variance 77/(1 + 1?) [Problem 24(i)], and it follows 
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A(x) = 9 —— $ 
T1 wW1 +17 ga 
1+7? —c¥1 +7? n 
» Sero 


ES 


-6 


Next let h(0) = V2mA(8) = e-* 2" and 
D- OL) dð < Janv [d(8){ PIX- c<0<X+cand XE C] 
-B[B(Q0 14 0]) d0 + 5. 


The integral on the right side is the difference of two integrals each of which equals 
P[X-c« 8 « X * cand XE C], and is therefore 0, so that D < ¢/r. 

Consider now a sequence of normal priors A, (f) with variances 1,3 > oo, and 
the corresponding sequences A,(8) and Dj. Then 0s D, S c/n, and hence 
D, > 0. On the other hand, D, is of the form D, = f£, ACD) hs, (0) d0, where 
A(9) is continuous, nonnegative, and > 0 for some f. There exists 8 > 0 such that 
A(0) > 14(8,) for |@ — 6j < 8 and hence 


D, > [Ahha (0) d0 BACH) = € m 
L 


This provides the desired contradiction. 


That also no negatively semirelevant subsets exist is a consequence of the 


following result. 
A Theorem 2. Let S(x) be a family 
alo € S(X)] = y for all 8, and suppose 
(i) If C is semirelevant, then its comp 
Opposite bias. 
(ii) If there exists a constant a such that 


of confidence sets for 8 such that 
that 0 < P(C) « 1 for all 0. 
lement C is semirelevant with 


1>P(C)>4>9 for all 8 


and C is relevant, then C is relevant with opposite bias. 
ate consequence of the 


[1 - (Oly ve 


à identity 
Proof. The result is an immedi ; 


P(C)[Ye(0) bi 7] = 
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The next example illustrates the situation in which a semirelevant subset 
exists but no relevant one. 


Example 10. Let X be N(0,1), and consider the uniformly most accurate lower 
confidence bounds 0 = X — c for 0, where ®(c) = y. Here S( X) is the interval 
[X — c, co) and it seems plausible that the conditional probability of 0 € S( X) will 
be lowered for a set C of the form X > k. In fact 


(c) — 6(k — 0) ES. 

ESO UE 0) Ejus 
Q9) A(X-cs9X2)e-| 1-e(k-s) "n ST 

0 when 0<k-—c, 


The probability (29) is always < y, and tends to y as 6 — oo. The set X >k is 
therefore semirelevant negatively biased for the confidence sets S(X). 

We shall now show that no relevant subset C with P,(C) > 0 exists in this case. 
It is enough to prove the result for negatively biased sets; the proof for positive bias 
is exactly analogous. Let A be the set of x-values —oo < x < c + 0, and suppose 
that C is negatively biased and relevant, so that 


PB[XeA4|C] <y-e€ fora 6. 


If 

a(0)=P,(XeEC), b(0)- P(XeAncC), 
then 
(30) b(0) s(y—«)a(0)  foralló. 


The result is proved by comparing the integrated coverage probabilities 
R R 
A(R) = 9)40, B(R) = 
(R) = f^ a(0) (R) = f" b(0) 48 
with the Lebesgue measure of the intersection C N (-R, R), 
R 
a(R) = I dx, 
(R) = Jf 1c(x) 

where Ic(x) is the indicator of C, and showing that 


3 A(R) oo BR) 
(31) a(R) A; a(R) ^Y as R> o. 


This contradicts the fact that by (30), 


B(R)s(v-c)A(R) forall R, 
and so proves the desired result. 
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To prove (31), suppose first that (œ) < co. Then if ¢ is the standard normal 
density 


Alo) = ICE = O)idew [4 = a(o), 


and analogously B(oo) = y#(co), which establishes (31). 
When (co) = co, (31) will be proved by showing that 


(2) — A(R)-4(R)* K(R), BCR) = yu (R) + KR), 


where K,(R) and K. 2(R) are bounded. To see (32), note that 


wR) =f" tela) de f iC f sm ao] ae 


T Fn ^ Te) 6(x ~ 0) dx] a8, 


while 
(33) A(R) = [^ [7 1: 6996-6 ax] a. 


A comparison of each of these double integrals with that over the region — R < x 
SR, -R«6«- R, shows that the difference A(R) — a(R) is made up of four 
Mtegrals, each of which can be seen to be bounded by using the fact that 
lItlé(r) dt < oo [Problem 24(ii)]. This completes the proof. 


Example 11. Let X... X be independently normally distributed as N(£, o?), 
m der the uniformly most accurate equivariant (and unbiased) confidence 
“vals for £ given by (28) of Chapter 6. ari d 
It was Shown by Buehler and Feddersen (1963) and Brown (1967) that in this 

there exist Positively biased relevant subsets of the form 


(34) € Hi 
S 


<k. 

Particular, for confidence level y=.5 and n=2, Brown shows oraa 

n 21/26 -X|s1iü4 V2), the conditional level is > 2 for all pucr lier 
dt follows from Theorem 2 that C is negatively biased semirelevant, an FE es 

ih 9) shows that any set C*: S <k has the same property. These re AS 
Mitively plausible, since the length of the confidence intervals is Laks ihe long 

ee Would expect short intervals to cover the true value less often 

prop corem 2 does not show that © is negatively biased 
an ility of the set (34) tends to zero as £/a — co. It was i sei 

"son (1976) that no negatively biased relevant subset exists in 


relevant, since the 
in fact proved by 
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The calculations for C throw some light on the common practice of stating 
confidence intervals for £ only when a preliminary test of H : = 0 rejects the 
hypothesis. For a discussion of this practice see Olshen (1973), and Meeks and 
D'Agostino (1983). 


The only type of example still missing is that of a positively biased 
relevant subset. It was pointed out by Fisher (1956a, b) that the Welch-Aspin 
solution of the Behrens-Fisher problem (discussed in Chapter 6, Section 6) 
provides an illustration of this possibility. The following are much simpler 
examples of both negatively and positively biased relevant subsets. 


Example 12. An extreme form of both positively and negatively biased subsets 
was encountered in Chapter 7, Section 11, where lower and upper confidence 
bounds A < A and A < A were obtained in (98) and (99) for the ratio A = oj/o? in 
a model II one-way classification. Since 


P(A<AJA<0)=1 and P(A<A\A<0) - 0, 


the sets C, : A < 0 and C,: T < 0 are relevant subsets with positive and negative 
bias respectively. 


The existence of conditioning sets C for which the conditional coverage 
probability of level-y confidence sets is 0 or 1, such as in Example 12 or 
Problems 27, 28 are an embarrassment to confidence theory, but fortunately 
they are rare. The significance of more general relevant subsets is less clear,* 
particularly when a number of such subsets are available. Especially awk- 
ward in this connection is the possibility [discussed by Buehler (1959)] of the 
existence of two relevant subsets C and C’ with nonempty intersection and 
opposite bias. 
> If a conditional confidence level is to be cited for some relevant subset C, 
it seems appropriate to take account also of the possibility that X may fall 
into C and to state in advance the three confidence coefficients y, yc, and 
Ye. The (unknown) probabilities P((C) and P,(C) should also be consid- 
ered. These points have been stressed by Kiefer, who has also suggested the 
extension to a partition of the sample space into more than two sets. For an 
account of these ideas see Kiefer (1977a, b), Brownie and Kiefer (1977), and 
Brown (1978). 

Kiefer's theory does not consider the choice of conditioning set Or 
statistic. The same question arose in Section 2 with respect to conditioning 
on ancillaries. The problem is similar to that of the choice of model. The 


answer depends on the context and purpose of the analysis, and must be 
determined from case to case. 


4 “For a discussion of this issue, see Buehler (1959), Robinson (1976, 1979a), and Bondar 
(1977). 


ith known probabilities p and q perform either é or 
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5. PROBLEMS 
Section 1 


. Let the experiments 4 and Æ consist in observing X: N(£,oj) and 


X: N(£, o7) respectively ( i 

, r 9, < o,), and let one of the two experiments be 
ner with P(£) = P(.5*) =}. For testing H:£ —0 against $ =, 
etermine values o), o}, £,, and a such that 


(i) a <a; (i) a> a, 


where the a, are defined by (9). 


. Under the assumptions of Problem 1, determine the most accurate invariant 


(under the transformation X’ = — X) confidence sets S( X) with 
P(E © S(X)|é) + P(E E S(X)|F = 2v. 


Find examples in which the conditional confidence coefficients yp given & and 
nh given F satisfy 


(i) <n: (G) w>: 


à la given by (3), (8), and (9) is most powerful under the stated assump- 


* Let X,,..., X, be independently distributed, each with probability p or q as 


NG, aj) or N(£, oè). 
G) k p» unknown, determine the UMP unbiased test of H : £ = 0 against 
:€>0. 
(i) Determine the most powerful test of H against the alternative ge 
is known that p = 1, and show that a UMP unbiased test does not exist 
ftn = this case. 

Gi) Let a, (k = 0,..., n) be the condition 
Powerful test of part (ii) given that k 

and n — k from N(£, oj ). Investigate 


al level of the unconditional most 
of the X's came from N(£, og) 
the possible values ap, 0, : «s €: 
F, with X distributed 
: 0 = 0 against 0 > 0 


as N(8,1) under € = ting H 
or N( — 6,1) under F. For testing dl leveka. test. These 


exist a UMP unconditional and a UMP condition 


Since 
incide and do not depend on the value of p. 
of X under & and F are 


Me preceding iti 
beð problem, suppose that the cam ~~ MP conditional and un- 


and (1/@)e~*/* i 
9" respectively. Compare 
tional tests of H: @ = 1 against K: 0 > 1. 
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Section 2 
Let X, Y be independently normally distributed as N(@,1), and let 
V-Y-X 
and 


w-[Y-X it X*Y»0, 
X-Y if X«Y«0. 


(i) Both V and W are ancillary, but neither is a function of the other. 
Gi) (V, W) is not ancillary. 
[Basu (1959). 


An experiment with m observations X,,..., X, is planned, with each X, 
distributed as N(0,1). However, some of the observations do not materialize 
(for example, some of the subjects die, move away, or turn out to be 
unsuitable). Let I -1o0r0as X, is observed or not, and suppose the I, are 
independent of the X's and of each other and that P(I, = 1) =p for all j. 


() If p is known, the effective sample size M — EI, is ancillary. 

(i) If p is unknown, there exists a UMP unbiased level-a test of H: 0 < 0 
vs. K:6 » 0. Its conditional level (given M = m) is a,, = a for all 
m=0,...,n. 


Consider n tosses with a biased die, for which the probabilities of 1,...,6 
points are given by 


1 2 3 4 5 6 
1-0 2-0 3-9 146 246 340 
12 12 12 12 12 12 


and let X, be the number of tosses showing i points. 


G) Show that the triple Z, = X, + X, Z=X% +X, Z=X +X isa 
maximal ancillary; determine its distribution and the distribution of 
X... X, given Z, = 2, Z, = 22, Z = 23. 

(Gi) Exhibit five other maximal ancillaries, 


[Basu (1964).] 
In the preceding problem, suppose the probabilities are given by 
1 2 3 4 5 6 
1-8 1:320, XOpo930 7 er ^1 P356 1+ 30 
6 6 6 6 6 6 


Exhibit two different maximal ancillaries. 
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ll. Let X be uniformly distributed on (0, 0 + 1), 0 < 6 < co, let [X] denote the 
largest integer < X, and let V = X — [X]. 
(i) The statistic V(X) is uniformly distributed on (0,1) and is therefore 
ancillary. 
(i) The marginal distribution of [ X] is given by 
[x] [?] with probability 1 — V(@), 
[9] +1 with probability V(6). 
(iii) Conditionally, given that V = v, [X] assigns probability 1 to the value 
[0] if V(@) < v and to the value [0] + 1 if V(@) > v. 
[Basu (1964).] 
12. Let X, Y have joint density 


P(x, y) = 2f(x)f( y) F(0x»), 


where f is a known probability density symmetric about 0, and F its 
cumulative distribution function. Then 
G) p(x, y) is a probability density. 
(ii) X and Y each have marginal density f and are therefore ancillary, but 
(X, Y) is not. 
(i) X- Y isa sufficient statistic for 0. 
[Dawid (1977).] 

13. A sample of size n is drawn with replacement from a population consisting of 
N distinct unknown values (a;,..., ay). The number of distinct values in the 
sample is ancillary. 

14. Assuming the distribution (22) of Chapter 4, Section 9, show that Z is 
S-ancillary for p = p,/(p, + p-). 

15. In the situation of Example 3, X + Y is binomial if and only if ^ — 1. 

16. In the situation of Example 2, the statistic Z remains S-ancillary when the 
parameter space is Q = {(A, n): p <A}. 

17. Suppose X = (U, Z), the density of X factors into 


Po.o( x) = (8, 8) go (us z)ho(z) k(u, 2), 


and the parameters 8, è are unrelated. To see that these assumptions are not 
enough to insure that Z is S-ancillary for 6, consider the joint density 


C(0,0)e- 30-9 36-9" r(u, 2), 
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where F(u, z) is the indicator of the set ((u, z): u s z). 
[Basu (1978).] 
Section 3 
Verify the density (16) of Example 7. 
Let the real-valued function f be defined on an open interval. 


(i) If f is logconvex, it is convex. 
Gi) If f is strongly unimodal, it is unimodal. 


. Let X, X, and Y,,..., Y, be positive, independent random variables 


distributed with densities /(x/o) and g(y/T) respectively. If f and g have 
monotone likelihood ratios in (x, e) and (y, r) respectively, there exists a 
UMP conditional test of H: r/o < A, against r/o > A, given the ancillary 
statistics U; = X/X, and V, = Y/Y, (i= 1,...,m—1; j= 1,..., - 1). 

Let V,,...,¥, be independently distributed as N(0,1), and given V, = 
yy cay Vy ™ Up, let X, (i = 1,..., n) be independently distributed as N(6v,, 1). 


(i) There does not exist a UMP test of H: 0 = 0 against K : 0 > 0. 
(ii) There does exist a UMP conditional test of H against K given the 
ancillary (V,,...,V,) 


[Buchler (1982).] 


. In Example 8, 


(i) the problem remains invariant under G' but not under G,; 
(ii) the statistic D is ancillary. 


Section 4 


. In Example 9, check directly that the set C = (x: x < —k or x > k} is nota 


[a ener 9 amie re for the confidence intervals (X ~ c, 
+c). 


G) Verify the posterior distribution of 6 given x claimed in Example 9. 
(ii) Complete the proof of (32). 


. Let X be a random variable with cumulative distribution function F. If 


E|X| < oo, then f°, F(x) dx and ffl — F(x)) dx are both finite. 
[Apply integration by parts to the two integrals.] 


Let X have probability density f(x — 8), and suppose that E|X| < oc. For the 
— X — c « 0. there exist semirelevant but no relevant sub- 


[Buehler (1959).] 
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28. 
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Let X,,..., X, be independently distributed according to the uniform distribu- 
tion U(0, 0 + 1). 


(i) 


(ii) 


(iii) 


Uniformly most accurate lower confidence bounds 6 for 0 at confidence 
level 1 — a exist and are given by 


9 = max( Xa, — k, Xu = 1), 


where Xay = min(X,,..., X,), X, = max(X,,..., X,), and (1 = k)" = 
a. 

The set C: x,,, — xq 2 1 — K is a relevant subset with Py(@ < 6|C) = 
1 for all 0. 

Determine the uniformly most accurate conditional lower confidence 
bounds (v) given the ancillary statistic V = X,,, — Xj, ** v, and com- 
pare them with 8. 


(The conditional distribution of Y = X4, given V = v is U(0,0 1 —v)] 
[Pratt (1961), Barnard (1976).] 


(i) 


(ii) 


(ii) 


Under the assumptions of the preceding problem, the uniformly most 
accurate unbiased (or invariant) confidence intervals for @ at confidence 


level 1 — a are 
8 = max( Xay + d, Xn) = 1 < 0 < min( Xy, Xu, = d) = Ë, 
where d is the solution of the equation 
2d"=a if a<1/2""', 
24 -(44-1)-a if a»1/2"'. 


The sets C, : Xin) 7 Xa) > d and C; : Xi, = Xy, © 2d = 1 are relevant 
subsets with coverage probability 


P,[0<0<B\c] =1 and n[9«9«96] = 0. 


Determine the uniformly most accurate unbiased (or invariant) condi- 
tional confidence intervals @(v) < @ < 6(v) given V = v at confidence 
level 1 — a, and compare (v), (v), and (v) — &(v) with the corre- 
sponding unconditional quantities. 


[Welch (1939), Pratt (1961), Kiefer (1977a).] 

Instead of conditioning the confidence sets Ô € S(X) on a set C, consider a 
randomized procedure which assigns to each point x a probability y(x) and 
makes the confidence statement Ô € S(x) with probability y(x) when x is 
observed.* 


*Randomized and nonrandomized conditioning is interpreted in terms of betting strategies 


by Buehler (1959) and Pierce (1973). 


564 CONDITIONAL INFERENCE [10.6 


(i) The randomized procedure can be represented by a nonrandomized 
conditioning set for the observations (X,U), where U is uniformly 
distributed on (0,1) and independent of X, by letting C = ((x, u): u < 
¥(x)}. 

(ii) Extend the definition of relevant and semirelevant subsets to randomized 
conditioning (without the use of U). 

(ii) Let @ € S( X) be equivalent to the statement X € A(0). Show that y is 
positively biased semirelevant if and only if the random variables (X) 
and 1,4, (X) are positively correlated, where J, denotes the indicator of 
the set A. 


30. The nonexistence of (i) semirelevant subsets in Example 9 and (ii) relevant 
subsets in Example 10 extends to randomized conditioning procedures. 


6. REFERENCES 


Conditioning on ancillary statistics was introduced by Fisher (1934, 1935, 
1936).* The idea was emphasized in Fisher (1956b) and by Cox (1958), who 
motivated it in terms of mixtures of experiments providing different amounts 
of information. The consequences of adopting a general principle of condi- 
tioning in mixture situations were explored by Birnbaum (1962) and Durbin 
(1970). Following Fisher's suggestion (1934), Pitman (1938) developed a 
theory of conditional tests and confidence intervals for location and scale 


The possibility of relevant subsets was pointed out by Fisher (19562, b). 
Its implications (in terms of betting procedures) were developed by Buehler 
(1959), who in particular introduced the distinction between relevant and 
semirelevant, positively and negatively biased subsets, and proved the 
nonexistence of relevant subsets in location models. The role of relevant 
subsets in statistical inference, and their relationship to Bayes and admissi- 
bility properties, was discussed by Pierce (1973), Robinson (1976, 19792, b), 
and Bondar (1977) among others. 

Fisher (1956a, b) introduced the idea of relevant subsets in the context of 
the Behrens-Fisher problem. As a criticism of the Welch-Aspin solution, he 
established the existence of negatively biased relevant subsets for that 
procedure. It was later shown by Robinson (1976) that no such subsets exist 
for Fisher's preferred solution, the so-called Behrens-Fisher intervals. This 
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*Fisher's contributions to this topic are discussed in Savage (1976, pp. 467-469). 
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ppendix 


1. EQUIVALENCE RELATIONS; GROUPS 


A relation: x ~ y among the points of a space F is an equivalence relation 
it is reflexive, symmetric, and transitive, that is, if 


G) x~ x forall x € 4; 

I (ii) x ~ y implies y ~ x; 

Gi) x ~ y, y ~ 2 implies x ~ z. 

. Example 1. Consider a class of statistical decision procedures as a space, of 

whicl the individual procedures are the points. Then the relation defined by 8 ~ 6° 
the procedures 8 and 6’ have the same risk function is an equivalence relation. As 

her example consider all real-valued functions defined over the real line as 

X nis of a space. Then f~ g if f(x) = g(x) ae. is an equivalence relation. 

- Given an equivalence relation, let D, denote the set of points of the space 

‘that are equivalent to x. Then D, = D, if x ~ y, and D, N D, = 0 other- 
se. Since by (i) each point of the space lies in at least one of the sets D,, it 

follows that these sets, the equivalence classes defined by the relation ~ , 

constitute a partition of the space. 

— A set G of elements is called a group if it satisfies the following 

conditions. 

(i) There is defined an operation, group multiplication, which with any 

two elements a, b € G associates an element c of G. The element c 

; is called the product of a and b and is denoted by ab. 

— (ii) Group multiplication obeys the associative law 
(ab)c = a(bc). 


(iii) There exists an element e € G, called the identity, such that 


ae-ea-a forall a€G. 
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(iv) For each element a € G, there exists an element a^! € G, its 
inverse, such that 
aa. =a a =e. 
Both the identity element and the inverse a^! of any element a can 
be shown to be unique. 

Example 2. The set of all n X n orthogonal matrices constitutes a group if 
matrix multiplication and inverse are taken as group multiplication and inverse 
respectively, and if the identity matrix is taken as the identity element of the group. 
With the same specification of the group operations, the class of all nonsingular 
n Xn matrices also forms a group. On the other hand, the class of all n x n 
matrices fails to satisfy condition (iv). 

If the elements of G are transformations of some space onto itself, with 
the group product ba defined as the result of applying first transformation a 
and following it by b, then G is called a transformation group. Assumption 
(ii) is then satisfied automatically. For any transformation group defined 
over a space Z the relation between points of X given by 


x~y if thereexists a € G such that y = ax 


is an equivalence relation. That it satisfies conditions (i) (ii), and (iii) 
required of an equivalence follows respectively from the defining properties 
(iii), (iv), and (i) of a group. 

Let € be any class of 1:1 transformations of a space, and let G be the 
class of all finite products aja#!...a*!, with a,,...,a,,€ €, m= 
1,2,..., where each of the exponents can be +1 or —1 and where the 
elements a, a5,... need not be distinct. Then it is easily checked that G is 
a group, and is in fact the smallest group containing €. 


2. CONVERGENCE OF DISTRIBUTIONS 


When studying convergence properties of functions it is frequently conveni- 
ent to consider a class of functions as a realization of an abstract space 7 
of points f in which convergence of a sequence f, to a limit f, denoted by 
f, > f, has been defined, 


Example 3. Let p be a measure over a measurable space (Z, x). 


(i) Let F be the class of integrable functions. Then f, converges to f in the 
mean if* 


(1) fM. -fidu > 0. 


*Here and in the examples that follow, the limit f is not unique. More specifically, if 
f, > f, then f, > g if and only if f — g (ae. p). Putting f~ g when f=g (ae. p) 
uniqueness can be obtained by working with the resulting equivalence classes of functions 
rather than with the functions themselves. 
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(ii) Let F be a uniformly bounded class of measurable functions. The sequence 
f, is said to converge to f weakly if 


(2) findu | pan 


for all functions p that are integrable p. 

(ii) Let F be the class of measurable functions. Then f, converges to f 
pointwise if 
(3) f(x)-f(x) ae. p. 

A subset F of F is dense in F if, given any f € £, there exists a 
sequence in F) having f as its limit point. A space F is separable if there 
exists a countable dense subset of F. A space F such that every sequence 
has a convergent subsequence whose limit point is in F is compact.* A 
space F is a metric space if for every pair of points f, g in Z there is 
defined a distance d( f, g) = 0 such that 


(i) d(f, g) = 0 if and only if f = g; 
G) df, g) = d(g f 
Gii) d(f, g) + d(g, h) = d( f, h) for all f. gh. 
The space is pseudometric if (i) is replaced by 
G) d(f, f) = 0 for all f EF. 
A pseudometric space can be converted into a metric space by introduc- 


ing the equivalence relation f~g if d(f,g)= 0. The equivalence classes 
F,G,... then constitute a metric space with respect to the distance D(F,G) 


= d( f, g) where f E F, g € G. ph j 
In any pseudometric space a natural convergence definition is obtained 


by putting f, > f if d(f,, f) > 0- 


Example 4. The space of integrable functions of Example 3(i becomes a 
pseudometric space if we put 


alf, g) = fif- edu 


and the induced convergence definition is that given by (1). 
Example 5. Let # be a family of probability distributions over (£^, 2). Then 
is a metric space with respect to the metric 


(4) d(P,Q) = sup| P(A) - Q(4)]. 
Aes. 


*The term compactness is more commonly used for an alternative concept, which coincides 
with the one given here in metric spaces. The distinguishing term sequential compactness is then 
sometimes given to the notion defined here. 
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Lemma 1. Jf F is a separable pseudometric space, then every subset of F 
is also separable. 


Proof, By assumption there exists a dense countable subset ( /,} of F. 


$e [100 <=), 


and let A be any subset of F. Select one element from each of the 
intersections A N S, , that is nonempty, and denote this countable collec- 
tion of elements by Ao. If a is any element of A and m any positive integer, 
there exists an element /,_ such that d(a, f, ) < 1/m. Therefore a belongs 
to Sm, n» the intersection A N Sn, „„_ is nonempty, and there exists therefore 
an element of Ag whose distance to a is < 2/m. This shows that A, is 
dense in A, and hence that A is separable. 


Lemma 2. A sequence f, of integrable functions converges to f in the mean 
if and only if 


(5) frt J, fdp uniformly for A eof. 
Proof. ‘That (1) implies (5) is obvious, since for all A € 


fiam ~ fran] fu 7 f\dp. 


Conversely, suppose that (5) holds, and denote by A, and A;, the set of 
points x for which f, (x) > f(x) and f,(x) < f(x) respectively. Then 


Jie tian - f, 0. 1) n - f n - f) da ^ 0. 


Lemma 3. A i 
piles i in Fora gai bounded functions converges to a 


(6) [fon J fa for all A with u( A) < «c. 


Proof. "That weak convergence implies (6) is seen by taking for p in (2) 
the indicator function of a set A, which is integrable if p( 4) < oc. Con- 
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versely (6) implies that (2) holds if p is any simple function s = Xa,/, with 
all the u(4,) < oo. Given any integrable function p, there exists, by the 
definition of the integral, such a simple function s for which {|p — s| dp < 
«/3M, where M is a bound on the |/['s. We then have 


Shto- dul +| fr- p) an] +| JU - o 


fu. - noa] 


The first two terms on the right-hand side are < «/3, and the third term 
tends to zero as n tends to infinity. Thus the left-hand side is < « for n 
sufficiently large, as was to be proved. 

Lemma 4.* Let f and fy, n = 1,2,..., be nonnegative integrable functions 
with 


fidu- [1 an - Y. 


Then pointwise convergence of f, to f implies that f, > f in the mean. 

Proof. Vg, f, — f, then g, 2 —f, and the negative part g, = 
max( — g,,0) satisfies |g, | s f. Since g,(x) — 0 (ac. p), it follows from 
Theorem 1(ii) of Chapter 2 that fg, dp — 0, and fg; dp then also tends 
to zero, since fg, dp = 0. Therefore f|g,| dn. f(g; + 8, ) dp — 0, as was 
to be proved. 

Let P and P, n= 1,2,..., be probability distributions over (Gr, v) 
with densities p, and p with respect to p. Consider the convergence 
definitions 

(a) pP, >p (ae. p); 

(b) fip, — Pl dp > 0; 

(c) [gp,dp —> [gp dp for all bounded measurable g; 

and 


(b) P,CA) — P(A) uniformly for all A € of; 
(c) P(A) — P(A) for all A € af. 
Then Lemmas 2 and 4 together with a slight modification of Lemma 3 


show that (a) implies (b) and (b) implies (c), and that (b) is equivalent to (b^) 
and (c) to (c^). It can further be shown that neither (a) and (b) nor (b) and 


(c) are equivalent. 


*Scheffé (1947). 
"Robbins, (1948). 
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3. DOMINATED FAMILIES OF DISTRIBUTIONS 


Let M be a family of measures defined over a measurable space (X, 7). 
Then æ is said to be dominated by a o-finite measure p defined over 
(£, 4) if each member of M is absolutely continuous with respect to p. 
The family Æ is said to be dominated if there exists a o-finite measure 
dominating it. Actually, if M is dominated there always exists a finite 
dominating measure. For suppose that is dominated by p and that 
4 — UA, with u(A,) finite for all i. If the sets A, are taken to be mutually 
exclusive, the measure »(4) = Lu(A N A,)/2'u(A,) also dominates .// and 
is finite. 


Theorem 1.* A family P of probability measures over a Euclidean space 
(4, &f) is dominated if and only if it is separable with respect to the metric (4) 
or equivalently with respect to the convergence definition 


P,>P if P(A)— P(A) uniformly for A €». 


Proof. Suppose first that P is separable and that the sequence ( P, } is 
dense in Y, and let y = XP,/2". Then (A) = 0 implies P,( 4) = 0 for all 
n, and hence P(A)= 0 for all P € 2. Conversely suppose that £^ is 
dominated by a measure u, which without loss of generality can be assumed 
to be finite. Then we must show that the set of integrable functions dP/dj. 
is separable with respect to the convergence definition (5) or, because of 
Lemma 2, with respect to convergence in the mean. It follows from Lemma 
1 that it suffices to prove this separability for the class . of all functions f 
that are integrable p. Since by the definition of the integral every integrable 
function can be approximated in the mean by simple functions, it is enough 
to prove this for the case that Z is the class of all simple integrable 
functions. Any simple function can be approximated in the mean by simple 
functions taking on only rational values, so that it is sufficient to prove 
separability of the class of functions Er,Z '4, Where the r’s are rational and 
the A's are Borel sets, with finite p-measure since the f ’s are integrable. It is 
therefore finally enough to take for F the class of functions Z4, which are 
indicator functions of Borel sets with finite measure. However, any such set 
can be approximated by finite unions of disjoint rectangles with rational 
end points. The class of all such unions is denumerable, and the associated 
indicator functions will therefore serve as the required countable dense 
subset of F. 


*Berger, (1951). 
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An examination of the proof shows that the Euclidean nature of the 
space (4, x) was used only to establish the existence of a countable 
number of sets A, € »£ such that for any A € 2 with finite measure there 
exists a subsequence A, with p(4,) > (A). This property holds quite 
generally for any o-field ^Y which has a countable number of generators, that 
is, for which there exists a countable number of sets B; such that is the 
smallest o-field containing the B,.t It follows that Theorem 1 holds for any 
o-field with this property. Statistical applications of such o-fields occur in 
sequential analysis, where the sample space 2 is the union Z= U,Z, of 
Borel subsets 2, of i-dimensional Euclidean space. In these problems, Z^, is 
the set of points (x,,..., x;) for which exactly i observations are taken. If 
wf, is the o-field of Borel subsets of Z, one can take for the o-field 
generated by the s, and since each s, possesses a countable number of 
generators, so does X. 

If Z does not possess a countable number of generators, a somewhat 
weaker conclusion can be asserted. Two families of measures M and MW are 
equivalent if p(A) = 0 for all p E Æ implies »(A) = 0 for all v € MN and 
vice versa. 


Theorem 2.* A family P of probability measures is dominated by a 
o-finite measure if and only if P has a countable equivalent subset. 


Proof. Suppose first that P has a countable equivalent subset 
{P,, P5,...). Then 2 is dominated by p = EP,/2". Conversely, let 2^ be 
dominated by a o-finite measure p, which without loss of generality can be 
assumed to be finite. Let 2 be the class of all probability measures Q of the 
form Xc,P;, where P, € 2, the c's are positive, and Ec, = 1. The class Lis 
also dominated by p, and we denote by q a fixed version of the density 
dQ/dy. We shall prove the fact, equivalent to the theorem, that there exists 
Qo in 2 such that Q,(A) = 0 implies Q(.) = 0 for all Q e 2. 

Consider the class € of sets C in o for which there exists Q € 2 such 
that q(x) > 0 a.e. p on C and Q(C) > 0. Let w(C,) tend to supy (C), let 
qx) > 0 a.e. on C;, and denote the union of the C, by Cy. Then qj(x) = 
Ye,q,(x) agrees a.e. with the density of Qo = Xc,Q, and is positive a.e. on 
Cy, so that Cy € €. Suppose now that Qo(A) = 0, let Q be any other 
member of 2, and let C = {x: q(x) > 0). Then Qo(4 N ©) = 0, and 
therefore p(A N Cy) = 0 and Q(4 N €) = 0. Also Q(A N C, n C) = 0. 
Finally, Q(4 N ČN C) » 0 would lead to p(C, U [4 n C, n C) > 
(Cy) and hence to a contradiction of the relation p(Cy) = supe p (C), since 
A n €, C and therefore Co U [4 N Co N C] belongs to €. 


tA proof of this is given for example by Halmos (1974, Theorem B of Section 40). 
*Halmos and Savage (1948). 
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4. THE WEAK COMPACTNESS THEOREM 


The following theorem forms the basis for proving the existence of most 
powerful tests, most stringent tests, and so on. 


Theorem 3.! (Weak compactness theorem.) Let y be a a-finite measure 
over a Euclidean space, or more generally over any measurable space (Z, of ) 
for which of has a countable number of generators. Then the set of measurable 
functions $ with 0 < $ < 1 is compact with respect to the weak convergence 
(2), 


Proof. Given any sequence {¢,}, we must prove the existence of a 
subsequence (@, ) and a function $ such that 


lim fo, pdp = [opdp 
for all integrable p. If u* is a finite measure equivalent to p, then p* is 
integrable p* if and only if p = (dp* /dp)p* is integrable p, and [9p dp = 
Íep* dp* for all $. We may therefore assume without loss of generality that 
p is finite. Let ( p,) be a sequence of p's which is dense in the p's with 
respect to convergence in the mean. The existence of such a sequence is 
guaranteed by Theorem 1 and the remark following it. If 


&, (p) = fe. pdn. 


the sequence $,(p) is bounded for each p. A subsequence ®,, can be 
extracted such that 6, (p,,) converges for each p,, by the following diago- 
nal process. Consider first the sequence of numbers (6,( p,)) which pos- 
Sesses a convergent subsequence ®,,( p,), 0, ( p,),... . Next the sequence 
SG) Palpa). has a convergent subsequence ®,( p;), $,:( P2)... - 
Continuing in this way, let n, = nj, n; = n, ny = n/^,.... Then n, < ^; 
<... and the sequence (4, ) converges for each p,,. It follows from the 
inequality 


If. - 4. )eds| <| flo, = $n) Pm du 


* 2 fip — Pal dn 


that ©, ( p) converges for all p. Denote its limit by ( p), and define a set 


"Banach (1932). The theorem is valid even without the assumption of a countable number 
of generators; see Nölle and Plachky (1967), and Aloaglu's theorem, given for example in 
Royden (1968, Chapter 10, Theorem 17). 
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function $* over by putting 
$*(4) = 9(1,). 


Then * is nonnegative and bounded, since for all A, ®*(A) < p(A). To 
see that it is also countably additive let A = U A, where the A, are disjoint. 
Then ©*( A) = lim ®¥(U A,) and 


m 


f. ondu- LOA) 
Ut-ik k 


-1 


sS 


fi n, du - LO*(A,) 
UA, 


&du- X OA) 


Uf mik kem*l 


+ 


Here the second term is to be taken as zero in the case of a finite sum 
A =U"_,A,, and otherwise does not exceed 24(Uf-+414,), which can be 
made arbitrarily small by taking m sufficiently large. For any fixed m the 
first term tends to zero as i tends to infinity. Thus ®* is a finite measure 
over (F, of). It is furthermore absolutely continuous with respect to p, 
since 4(A) = 0 implies ®, (74) = 0 for all i, and therefore ®(/,) = *(A) 
= 0. We can now apply the Radon-Nikodym theorem to get 


$*(A) = f o4 for all A, 
A 
with 0 < $ < 1. We then have 
J du ^ [dn for all A, 
wu A 
and weak convergence of the ¢,, to 9 follows from Lemma 3. 
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Nikodym derivative 
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test for, 392 

Admissibility, 17; Bayes method for proving, 
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families, 307; of invariant procedures, 28, 
311; of multiple comparison procedures, 
384: of UMP invariant tests, 305; of UMP 
unbiased tests, 170; of unbiased procedures, 
21, 305. See also Alpha-admissibility ; 
d-admissibility; Inadmissibility 

a. e., see Almost everywhere 

Aggregation (of several contingency tables), 
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Almost everywhere (a. e.), 40, 140 

Almost invariance: of decision procedures, 24; 
of likelihood ratio, 341; relation to 
invariance, 297, 298, 316, 340; relation to 
invariance of power function, 300; relation 
to maximin tests, 516; relation to 
unbiasedness, 302; of sets, 342; of tests, 
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Alpha-admissibility, 306, 342, 384 
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Analysis of variance, 375, 395, 444, 446; 
different models for, 418; for one-way 
classification, 375; in random effects 
model, 425; robustness of F-tests, 401; 


for two-way classification, 390, 395. See 
also Linear hypothesis; Linear model 

Ancillary statistic, 542, 560, 564, 565, 566; 
and invariance, 543; maximal, 545, 560; 
and sufficiency, 545. See also Partial 
ancillarity 

Approximate hypotheses: extended Neyman- 
Pearson lemma for, 512, 515 

Arcsine transformation for binomial variables, 
432, 445 

Association, 162; spurious, 162; Yule’s 
measure of, 157. See also Dependence, 
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Asymptotic (relative) efficiency, 321 

Asymptotic normality: of functions of 
asymptotically normal variables, 205; of 
mean, 204. See also Central limit theorem 

Asymptotic optimality, vii, 477, 485 

Attributes: paired comparisons by, 169, 291, 
510, 526; sample inspection by, 80, 293 

Autoregressive process (first order), 212 

Average power, maximum, 429 


Bartlett's test for variances, 378 

Basu’s theorem, 191 
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Bayesian inference, 15, 70, 227, 427, 465, 
511, 564 

Bayes risk, 14 

Bayes solution, 14, 18, 25, 33; to maximize 
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Prior distribution 

Bayes sufficiency, 21, 22, 31 

Bayes test, 125, 343, 430, 465, 498 
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Behrens-Fisher problem, 209, 262, 304, 360, 
361, 558, 564, 566; for many samples, 379; 
multivariate, 462; nonparametric, 323. 

See also Welch-Aspin test 

Beta distribution, 200, 272; as distribution of 
order statistics, 345; noncentral, 369, 428; 
relation to F-distribution, 200; relation to 
gamma distribution, 272; in testing linear 
hypotheses, 369; in testing ratio of variances, 
200, 255 

Bimeasurable transformation, 284 

Binomial distribution b(p,n), 2; in comparing 
two Poisson distributions, 153; completeness 
of, 141; as exponential family, 56, 81; as 
log-linear model in bio-assay, 178; variance 
stabilizing transformation for, 432, 445. 
See also Contingency tables; Multinomial 
distribution; Negative binomial 
distribution Nb; Two by two table 

Binomial probabilities: comparison of two, 

121, 154, 159, 161, 175, 180, 183, 261; 
confidence bounds for, 93, 117; confidence 
intervals for, 219, 221; credible region for, 
227; one-sided test for, 93, 113, 167; two- 
sided test for, 118, 138, 167, 171. See also 
Contingency tables; Independence, test for; 
Median; Paired comparisons; Sample 
inspection; Sign test 

Binomial trials, 7; obtained by dichotomizing 
continuous variables, 164; sufficient 
Statistics for, 19, 28. See also Inverse 
sampling 

Bioassay, 178 

Bivariate distribution(general): class of one- 
parametric families of, 251; testing for 
independence or lack of correlation in, 
250, 350. See also Dependence, positive 

Bivariate normal correlation coefficient: 
confidence bounds for, 353; distribution of, 
267, 270; test for, 249, 304, 340 

Bivariate normal distribution, 249, 267, 271; 
ancillary statistics in, 545; joint distribution 
of second moments in, 268; test for 
independence in, 249, 253, 271; testing 
parameters in, 268, 305 

Borel set, 35 

Bounded completeness, 144, 172, 191, 300; 
example of, without completeness, 173. See 
also Completeness of family of distributions 


Canonical form: for model II two-way layout, 
438, 441; for multivariate linear hypothesis, 
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454; for multivariate linear hypothesis with 
covariates, 471; for nested classification in 
model II, 423, 438; for repeated 
measurement model, 467; for univariate 
linear hypothesis, 366, 370 

Cartesian product, 40 

Cauchy distribution, 86, 115, 510, 567 

Causal influence, 162 

CDF, see Cumulative distribution function 

Center of symmetry: confidence intervals for, 
263. See also Symmetry 

Central limit theorem, 204; for dependent 
variables, 213; Lindeberg form of, 402 

Chebyshev inequality, 257 

Chi-squared distribution, 56, 139; in estimating 
normal variance, 218, 229; as exponential 
family, 56; as limit for likelihood ratio, 487; 
in multivariate distribution theory, 490; 
non-central, 427, 428, 434, 447, 500; 
relation to beta-distribution, 200; relation 
to exponential distribution, 64, 82, 114; 
relation to F-distribution, 199; relation to 
t-distribution, 196; for testing linear 
hypotheses with known variance or 
covariance matrix, 431, 477; in testing 
normal variance, 110, 139, 194, 290; for 
total waiting time in Poisson process, 92. 
See also Gamma distribution; Normal one- 
sample problem, the variance; Wishart 
distribution 

Chi-squared test, 477, 480, 500, 502; 
restricted, 481, 500, 501; in r X c 
contingency tables, 487; for testing 
goodness of fit, 480, 494; for testing 
uniform distribution, 480, 482 

Cluster sampling, 211 

Cochran-Mantel-Haenszel test, 165 

Coefficient of variation, 549; confidence 
bounds for, 352, 356; tests for, 294, 303 

Comparison of experiments, 86, 114, 116, 159, 
167, 223, 264, 339 

Completeness of a class of decision 
procedures, 17, 18; of classes of one-sided 
tests, 82, 83, 461; of class of two-sided tests, 
172; relation to sufficiency, 64. See also 
Admissibility 

Completeness of family of distributions, 141, 
172, 173, 180; of binomial distributions, 
141; for exponential distributions, 256; of 
exponential families, 142; of normal 
distributions, 142, 172; of order statistics, 
163, 173, 183, 187; relations to bounded 
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completeness, 144, 173; of uniform 
distributions, 141, 172 

Completion of measure, 35 

Complexity: of multiple comparison 
procedure, 387 

Components of variance, 425, 558. See also 
Random effects model 

Composite hypothesis, 72; large-sample tests 
for, 483; vs. simple alternative, 104 

Conditional distribution, 48; in bivariate 
normal distributions, 267; example of 
nonexistence, 48, 67; in exponential 
families, 58, 146; in Poisson distribution, 
65 

Conditional expectation, 44, 47, 50 

Conditional independence, 162; test of, 163 

Conditional inference, ix, 541, 558, 564, 566 

Conditionality principle, weak, 548 

Conditional power, 151, 170, 246, 541, 547 

Conditional probability, 43, 47, 48, 66 

Conditional test, 182, 549; most powerful, 
540, 543 

Confidence bands: for cumulative distribution 
function, 334, 354; in linear models, 406; 
for regression line, 417, 444; for regression 
surface, 444, See also Simultaneous 
confidence intervals 

Confidence bounds, 89; impossible, 421, 558; 
with minimum risk, 117; in monotone 
likelihood ratio families, 91; in presence of 
nuisance parameters, 213; randomized, 93; 
relation to median unbiased estimates, 95, 
214; relation to one-sided tests, 214; 
standard, 96, 229; uniformly most 
accurate, 90 

Confidence coefficient, 90, 213; conditional, 
558 

Confidence ellipsoids, 461, 490 

Confidence intervals, ix, 68, 94; of bounded 
length, 258, 259; for center of symmetry, 
263; distribution-free, 247, 263, 329; 
empty, 421, 558; history of, 126; 
interpretation of, 214, 225; logarithmically 
shortest, 331; loss functions for, 6, 24, 94, 
95; minimax, 524; for parameters suggested 
by data, 410; in randomization models, 247; 
randomized, 219; unbiased, 13, 24, 217. 
See also Simultaneous confidence intervals 

Confidence level, 89 

Confidence sets, 90; admissibility of, 313; 
average smallest, 330; conditional, 541; 
derived from a pivotal quantity, 333, 357; 
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equivariant, 327, 333, 524; example of 
inadmissible, 525; minimax, 524; relation 
with tests, 90, 214, 216; of smallest 
Lebesgue measure, 261, 330, 524; unbiased, 
217; which are not intervals, 225. See also 
Credible region; Equivariant confidence 
sets; Relevant and semirelevant subsets; 
Simultaneous confidence sets 

Conservative test, 155 

Consistency of sequence of tests, 356, 478, 
494 

Consumer preferences, 166, 167 

Contingency tables: general, 165; loglinear 
models for, 165; models for, 161, 495; 
r X c tables, 156, 487, 495; three factor, 
162; 2 X 2 X K, 162, 165, 179; 
2X2X2 XL, 179. See also Two by 
two tables 

Continuity correction, 155 

Contrasts, 388, 415; in multivariate case, 472, 
494 

Convergence: in law, 204; in mean, 570; 
pointwise, 571; in probability, 257; weak, 
571 

Convergence theorem: for densities, 5725 
dominated, 39; for functions of random 
variables, 205; monotone, 39. See also 
Cramér-Wold theorem 

Correlation coefficient: in bivariate normal 
distribution, 249; confidence bounds for, 
353; intraclass, 438; testing value of, 249, 
304, 340. See also Bivariate distribution; 
Dependence, positive; Multiple correlation 
coefficient; Rank correlation coefficient; 
Sample correlation coefficient R 

Countable additivity, 34 

Countable generators of o-field, 575 

Counting measure, 35 

Covariance matrix, 453; estimation of, 488; 
special structure, 440, 441; tests for, 379, 
462 

Covariates, 470, 552 

Cramér-Wold theorem, 491 

Credible region, 226; equal tails, 229; highest 
probability density, 227, 262 

Critical function, 71 

Critical region, 68 

Cross product ratio, see Odds ratio 

Cumulative distribution function (cdf), 36, 62; 
confidence bands for, 334, 354; empirical, 
323, 335; inverse of, 344. See also 
Kolmogorov test for goodness of fit 
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d-admissibility, 306, 342. See also 
Admissibility 

Data Snooping, 410, 476 

Decision problem: specification of, 2 

Decision space, 2, 3 

Decision theory, 29, 33; and inference, 4, 5,71 

Deficiency, 197 

Dependence, positive, 157, 176, 210, 251, 
271, 315, 350; measures of, 157. See also 
Correlation coefficient; Independence 

Design of experiments, 7, 8, 159, 396, 447. 
See also Random assignment; Sample size 

Directional error, 387 

Direct product, 40 

Dirichlet distribution, 262 

Distribution, see the following families of 
distributions: Beta, Binomial, Bivariate 
normal, Cauchy, Chi-squared, Dirichlet, 
Double exponential, Exponential, F, 
Gamma, Hypergeometric, Inverse 
Gaussian, Logistic, Multinomial, 
Multivariate normal, Negative binomial, 
Noncentral, Normal, Pareto, Poisson, 
Polya, t, Hotelling's T?, Triangular, 
Uniform, Weibull, Wishart. See also 
Exponential family; Monotone likelihood 
ratio; Total positivity; Variation diminishing 

Dominated convergence theorem, 39 

Dominated family of distributions, 53, 574, 
575 

Domination: of one procedure over another, 
17, See also Admissibility; Inadmissibility 

Double exponential distribution, 355, 509, 
567; locally most powerful test in, 531; 
UMP conditional test in, 550 

Duncan multiple comparison procedure, 
383, 385 

Dunnett’s multiple comparison method, 443 


EDF, see Empirical distribution function 

Efficiency, relative asymptotic, 321 

Efficiency robustness, 208, 322. See also 
Robustness 

Empirical distribution function(EDF), 323, 
335 

Envelope power function, 341, 525. See also 
Most stringent test 

Equivalence: of family of distributions or 
measures, 54, 575; of statistics, 43; of two 
measures, 61 

Equivalence classes, 569 

Equivalence relation, 569 


Equivariance, 12, 544, See also Invariance 

Equivariant confidence bands, 335, 406, 417, 
472 

Equivariant confidence sets, 327, 330; and 
pivotal quantities, 333, 357. See also 
Uniformly most accurate confidence sets 

Error of first and second kind, 69, 70 

Error rate per experiment, 388 

Essentially complete class, 18, 64, 82, 113. 
See also Completeness of a class of decision 
procedures 

Estimation, see Confidence bands; Confidence 
bounds; Confidence intervals; Confidence 
sets; Equivariance; Maximum likelihood; 
Median: Point estimation; Unbiasedness 

Euclidean sample space, 49 

Expectation (of a random variable), 38; 
conditional, 44, 47, 50 

Expected normal order statistics, 318 

Experimental design, see Design of 
experiments 

Exponential distribution, 23, 360; 
completeness in, 256; confidence bounds 
and intervals in, 92, 261, 354; order 
Statistics from, 65; other tests for, 355; 
relation to Pareto distribution, 123; relation 
to Poisson process, 23, 65, 82, 154; r- 
sample problem for, 354, 364; sufficient 
Statistics in, 28; testing against gamma 
distribution, 272; testing against normal or 
uniform distribution, 355; tests in, 93, 112, 
255; two-sample problem for, 338. See also 
Chi-squared distribution; Gamma 
distribution; Life testing 

Exponential family, 56, 59, 66; admissibility 
of tests in, 307; completeness of, 142; 
equivalent forms for, 150; median unbiased 
estimators in, 214; moments of sufficient 
statistics, 66; monotone likelihood ratio of, 
80, 119; natural parameter space of, 
57, 66; testing in multiparameter, 145, 171, 
181, 188; testing in one-parameter, 80, 120, 
135, 172; total positivity of, 119. See also 
One-parameter exponential family 

Exponential waiting times, 23, 65, 82, 92. See 
also Exponential distribution 


Factorization criterion for sufficient statistics, 
19, 30, 31, 55, 66, 67 

F-distribution, 199, 446, 449; in confidence 
intervals for ratio of variances, 219, 421; in 
Hotelling’s T2-test, 459; noncentral, 428; 
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relation to beta distribution, 200; relation to 
distribution of multiple correlation 
coefficient, 497; for simultaneous confidence 
sets, 475. See also F-test for linear 
hypothesis; F-test for ratio of variances 

Fiducial probability, 127, 131, 133, 229; 
distribution, 129, 229, 230 

Field, 60 

Finite decision problem, 64 

Fisher’s exact test, 155, 158, 180, 187. See 
also Two by two tables 

Fisher’s least significant difference, 382, 386 

Fixed effects model, 418. See also Linear 
model; Model I and II 

Free Group, 26 

Friedman’s rank test, 392 

F-test for linear hypothesis, 369; admissibility 
of, 370; as Bayes test, 430; has best average 
power, 429; in Fisher's least significant 
difference method, 382; in Gabriel's 
simultaneous test procedure, 382, 416; in 
mixed models, 426; permutation version of, 
450; power of, 369; robustness of, 378,379, 
401. See also F -distribution 

F-test for ratio of variances, 122, 199; 
admissibility of, 313; in mixed models, 426; 
in model II analysis of variance, 420, 424; 
nonrobustness of, 207, 378; power of, 200. 
See also F-distribution; Normal two- 
sample problem, ratio of variances 

Fubini's theorem, 40 

Fully informative statistics, 113 

Fundamental lemma, see Neyman-Pearson 
fundamental lemma 


Gabriel's simultaneous test procedure, 382, 
416 

Gamma distribution T(g, b), 123, 271, 272, 
356. See also Beta distribution; Chi- 
squared distribution; Exponential 
distribution 

Goodness of fit, 336, 355, 480, 482, 494. See 
also separate families 

Group, 569; amenable, 522; finite, 518; free, 
26; generated by subgroups, 288; linear, 
286, 299, 522; orthogonal, 286, 522, 525; 
permutation, 286, 298, 356; of rigid motions, 
525; scale, 285, 337; transitive, 285, 543, 
550; transformation, 282, 570; translation, 
285, 521; triangular, 305. See also 
Equivariance; Invariance 

Group family, 543, 550 
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Guaranteed power: achieved through 
sequential procedure, 151, 153, 260; with 
minimal sample size, 505 


Haar measure, 299 

Homogeneity, tests of: against ordered 
alternatives, 380; for exponential 
distributions, 364; for K two-by-two tables, 
165; for multinomial distributions, 495, 496; 
for multivariate normal means, 463; 
nonparametric, 380, 392; for normal means, 
374, 378, 379, 381, 389, 394; for normal 
variances, 376; for subsets of means, 381. 
See also Multiple comparisons; Normal 
many-sample problem 

Hotelling's T?-distribution, 459, 500; 
derivation of, 489; noncentral, 460, 500; 
x?-limit of, 490 

Hotelling’s T?-test, 459, 460, 500; 
admissibility of, 460, 498, 523; application 
to one- and two-sample problems, 459, 461, 
462, 471; application to two-factor mixed 
model, 466; as Bayes solution, 498; best 
average power of, 500; minimaxity of, 523; 
in multivariate regression, 462, 490; in 
repeated measurements, 466, 469; 
robustness of, 460, 462 

HPD (Highest probability density) credible 
region, 227, 262 

Huber condition(for robustness), 404, 436, 
448 

Hunt-Stein theorem, 519 

Hypergeometric distribution, 80; monotone 
likelihood ratio of, 80; relation to 
distribution of runs, 177; in testing equality 
of two binomials, 155; in testing for 
independence in a two by two table, 158, 
161; UMP one-sided test for testing mean 
of, 80. See also Fisher's exact test; Two by 
two tables 

Hypothesis testing, 3, 68; conditional, 539; 
history of, 126, 131; large-sample approach, 
ix, 477; loss functions for, 72, 82, 172, 292; 
without stochastic basis, 162 


Improper prior distribution, 226 

Inadmissibility, 17; of confidence sets for 
vector means, 525; of likelihood ratio test, 
341; of UMP invariant test, 305. See also 
Admissibility 

Independence: conditional, 162; of normal 
correlation coefficient from sample means 


592 


Independence (Continued ) 
and variances, 192; relation to absence of 
correlation, 250; of sample mean from 
function of differences in normal samples, 
191; of statistic from a complete sufficient 
Statistic, 191; of sum and ratio of independent 
x? variables, 192; of two random variables, 
40 

Independence, test for: in bivariate normal 
distribution, 248; in multivariate normal 
distribution, 462, 496; in nonparametric 
models, 251, 314, 350; in r X c contingency 
tables, 487; vs. tests for absence of 
correlation, 250; in two by two tables, 
156, 161 

Indicator function of a set, 39 

Indifference zone, 505 

Inference, statistical, 1, 4, 71. See also 
Decision theory 

Integrable function, 38 

Integration, 37 

Interaction, 393, 396, 444; in random effects 
and mixed models, 440, 441; test for absence 
of, 392, 394, 434 

Interval estimation, see Confidence 
intervals 

Into, see Transformation 

Intraclass correlation coefficient, 438 

Invariance: of decision procedure, 11, 12, 31, 
32; of likelihood ratio, 341; of measure, 299, 
518, 519; of power functions, 299, 300; 
relation to equivariance, 12; relation to 
minimax principle, 26, 516, 519; relation 
to sufficiency, 290, 301; relation to 
unbiasedness, 24, 302; of test, 284, 357; 
warning against inappropriate use of, 377. 
See also Almost invariance; Equivariance 

Invariant measure, 299, 518, 519; over 
orthogonal group, 518; over translation 
group, 521 

Inverse Gaussian distribution, 124, 272 

Inverse sampling: for binomial trials, 81; for 
Poisson variables, 82. See also Negative 
binomial distribution Nb; Poisson process 


Kendall's t-statistic, 351 

Kolmogorov test for goodness of fit, 336, 356, 
480, 494. See also Goodness of fit 

Kruskal-Wallis test, 380 


Large-sample tests, ix, 204, 380, 477, 480, 
503; for composite hypotheses, 483 


SUBJECT INDEX 


Latin square design, 396, 434 

Lawley-Hotelling trace test, 463; robustness 
of, 465; simultaneous confidence intervals 
based on, 471 

Least favorable distribution, 18, 104, 107, 
506, 510, 512, 516, 519 

Least squares estimates, 370, 374 

Lebesgue convergence theorems, 39 

Lebesgue integral, 38 

Lebesgue measure, 35 

Level of significance, see Significance level 

Life testing, 65, 114. See also Exponential 
distribution; Poisson process 

Likelihood, 16. See also Maximum 
likelihood 

Likelihood principle, 565 

Likelihood ratio: censored, 513; invariance 
of, 341; large-sample theory of, 486, 
503; preference order based on, 73, 79; 
procedure, 16; sufficiency of, 63 

Likelihood ratio test, 16, 126; example of 
inadmissible, 341; large-sample theory 
of, 486, 503 

Lindley's Paradox, 125 

Linear hypothesis, multivariate, 453, 465, 
498; Bayesian treatment of, 465; canonical 
form of, 454, 500; concerning row vectors of 
a matrix of means, 467, 470; with covariates, 
470; invariant test for when r — 1, 459; 
with known covariance matrix, 477; 
reduction through invariance of, 456, 488; 
robustness of tests for, 491; suggested by the 
data, 476; tests for when r > 1,463. Seealso 
Hotelling's T2-test; Multivariate analysis 
of variance (MANOVA ); Multivariate 
normal distribution; Multivariate one- 
sample problem; Multivariate two-sample 
problem; Regression, multivariate; 
Repeated measurements 

Linear hypothesis, univariate, 365, 449; 
admissibility of test for, 370; canonical 
form for, 366; inhomogeneous form of, 372; 
with known variance, 431; more efficient 
tests for, 380; parametric form of, 373; 
power of test for, 369; properties of test 
for, 369, 429, 522, 529, 538; reduction of, 
through invariance, 367; robustness of test 
for, 378, 379, 401; suggested by the data, 
411. See also Analysis of variance; 
Homogeneity, tests of; Mixed model; 
Model I and II; One-way classification; 
Regression; Two-way classification 
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Linear model, 365, 444; Bayesian inference 
for, 427; confidence intervals in, 391, 430; 
simultaneous confidence intervals in, 406, 
411, 417; testing set of linear functions in, 
483. See also Simultaneous confidence 
intervals and sets 

Locally optimal tests, 186, 507, 527, 528, 
529, 535, 538 

Location families, 84, 543; comparing two, 
289; conditional inference in, 543, 550, 
564, 566; condition for monotone likelihood 
ratio, 509; dichotomization of, 164; 
example lacking monotone likelihood ratio, 
86; existence of semi-relevant but not of 
relevant subsets for, 562, 567; are 
stochastically increasing, 84 

Location-scale families, 11, 32; comparing 
two, 338, 355. See also Normality, testing 
for 

Logistic distribution, 164, 165, 318, 320, 510, 
550, 567 

Logistic response model, 165 

Loglinear model, 165, 178 

Loss function, 1, 28; in confidence estimation, 
6, 24, 90, 94, 95; in hypothesis testing, 
72, 82, 172, 292; monotone, 95; 
specification of, 5 

L-unbiased, 13. See also Unbiasedness 


McNemar’s test, 169, 180 

Main effects, 389, 396, 433; confidence sets 
for, 391; tests for, 390, 394, 395. See also 
Two-way classification 

Mantel-Haenszel test, 165 

Markov chain, 176 

Markov property, 176 

Matched pairs: by attributes, 169, 179, 291, 
510, 526; comparison with complete 
randomization, 180, 264; confidence 
intervals for, 246, 264; generalization of, 
241; normal theory and permutation 
tests for, 239, 264; rank tests for, 
314, 323 

Maximal invariant, 285; ancillarity of, 543; 
distribution of, 289; method for 
determining, 287; obtained in steps, 287, 
288 

Maximin test, 505, 512, 515; existence of, 


527; local, 507; relation to invariance, 516, 


519, 533. See also Least favorable 
distribution; Minimax principle; Most 
stringent test 
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Maximum likelihood, 16, 17, 30, 31, 485, 
495. See also Likelihood ratio test 

Maximum modulus confidence intervals, 411 

Measurable: function, 36, 42; set, 35; space, 
35; transformation, 36 

Measure theory, xiii, 34, 66 

Median, 23; confidence bounds for, 120, 133; 
test for, 187, 530 

Median unbiasedness, 23, 29; examples of, 
216, 219; relation to confidence bounds, 
95, 214 

Metric space, 571 

Minimal complete class of decision procedures, 
17. See also Completeness of family of 
distributions; Essentially complete class 

Minimal sufficient statistic, 22, 28, 66 

Minimax principle, 14, 18, 32, 33, 535; in 
confidence estimation, 524; in hypothesis 
testing, 505; relation to invariance, 26, 516, 
519; relation to unbiasedness, 26, 507. See 
also Maximin test; Restricted Bayes 
solution 

Mixed model, 418, 427; for nested 
classification, 425; for two-way layout, 
427, 439, 440, 441. See also Model I and IL 

Mixtures of experiments, 539, 542, 559, 564 

MLR, see Monotone likelihood ratio 

Model I and II, 418, 446, 452. See also Fixed 
effects model; Mixed model; Random 
effects model 

Model selection, 10 

Monotone class of sets, 60 

Monotone convergence theorem, 39 

Monotone likelihood ratio, 78, 130; 
approximate, 516; conditional tests based 
on samples from a distribution with, 549, 
550, 551, 562; conditions for, 114; of 
distribution of correlation coefficient, 340; 
of exponential family, 80, 120; of 
hypergeometric distribution, 80; 
implications of, 85, 103, 115; of location 
parameter families, 104, 1 15,509; mixtures 
of families with, 530, 549, 551; of 
noncentral t, 295; of noncentral x? and F, 
428; relation to total positivity, 119; tests 
and confidence procedures in the presence 
of, 78, 82, 91 

Most stringent test, 358, 525, 538; existence 
of, 533 

Moving average process, 211 

Multinomial distribution, 56; as conditional 
distribution, 65; Dirichlet prior for, 262; for 
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Multinomial distribution (Continued ) 
entries of 2 X 2 table, 157, 169; limit 
distribution of, 479; in testing consumer 
preferences, 166; for 2 X 2 X K table, 162 

Multinomial model; maximum likelihood 
estimation in, 495; for r X c table, 487; 
testing a composite hypothesis in, 483; 
testing a simple hypothesis in, 478, 481; for 
three-factor contingency table, 162, 163; for 
2 X 2 table, 157, 159, 161, 169. See also 
Chi-squared test; Contingency tables 

Multiple comparisons, 4, 380, 396, 446, 451; 
complexity of, 387; significance levels for, 
382. See also Duncan and Dunnett multiple 
comparison methods; Newman-Keuls 
multiple comparison procedure; 
Simultaneous confidence intervals; Tukey 
levels; Tukey's T-method 

Multiple correlation coefficient, 497; 
distribution of, 446, 497, 500; optimum 
test for, 497, 503, 538 

Multiple decision procedures, 4, 27. See also 
Multiple comparisons; Three-decision 
problems 

Multivariate analysis of variance 
(MANOVA), 462. See also Linear 
hypothesis, multivariate 

Multivariate linear hypothesis, see Linear 
hypothesis, multivariate 

Multivariate normal distribution, 440, 441, 
453; as limit of multinomial distributions, 
479. See also Bivariate normal distribution 

Multivariate (normal) one-sample problem: 
simultaneous confidence sets in, 494; 
testing the covariance matrix, 462; testing 
independence of two sets of variates in, 496; 
testing the mean vector, 459, 466, 523. See 
also Hotelling's T2-test; Simultaneous 
confidence ellipsoids; Simultaneous 
confidence sets 

Multivariate (normal) two-sample problem, 
461, 532; Behrens-Fisher problem, 462; 
with covariates, 470, 552; robustness of 
tests for, 490; simultaneous confidence 
sets in, 494 

Multivariate regression, 462, 490, 496 

Multivariate t-distribution, 353 


Natural parameterspace of an exponential 
family, 57, 66 
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Negative binomial distribution Nb(p,m), 22, 
81, 181 

Neighborhood model, 512, 515, 516 

Nested classification, 422, 438 

Newman-Keuls multiple comparison 
procedure, 382, 386 

Newton’s identities, 47 

Neyman-Pearson fundamental lemma, 74, 
131; approximate version of, 512; censored 
version of, 513; generalized, 77, 96, 118, 
128 

Neyman structure, 141, 144 

Noncentral: beta distribution, 369, 428, 447, 
500; F-distribution, 426, 428, 429, 446; 
t-distribution, 196, 253, 276, 295, 303; x2- 
distribution, 427, 428, 429, 434, 447, 500 

Noninformative prior, 226 

Nonparametric: alternative approach to, 380; 
independence problem, 252, 317; many- 
sample problem, 380; one-sample problem, 
143, 263; test, 107; test in two-way layout, 
392. See also Permutation test; Rank tests; 
Sign test 

Nonparametric two-sample problem, 232, 
317; confidence intervals in, 246, 263, 347, 
362; omnibus alternatives, 322; universally 
unbiased test in, 348. See also Normal 
Scores test; Wilcoxon two-sample test 

Normal distribution N(Z,2), 3, 56; 
tests of, 355; testing against Cauchy, 
double exponential, exponential, or uniform 
distribution, 355. See also Bivariate normal 
distribution; Multivariate normal 
distribution 

Normality, testing for, 355. See also Normal 
distribution 

Normal many-sample problem: confidence 
sets for vector means; 331, 332, 406, 409, 
525, 535; tests for means in, 374, 377, 378, 
532, 548; tests for variances in, 376, 378. 
See also Homogeneity, tests of 

Normal one-sample problem, the coefficient 
of variation: confidence intervals for, 352, 
356; test for, 294, 303 

Normal one-sample problem, the mean: 
admissibility of test for, 309, 310; 
confidence intervals for, 215, 329, 554, 
557; credible region for, 226, 228; 
likelihood ratio test for, 108; median 
unbiased estimate of, 216; nonexistence of 
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test with controlled power, 253;nonexistence 
of UMP test for, 1 1 1; optimum test for, 111, 
195, 254, 255, 294, 303, 339, 372,549; test 
for, based on random sample size, 112; 
two-stage confidence intervals for, of 
fixed length, 259; two-stage test for, with 
controlled power, 260. See also Matched 
pairs; t-test 

Normal one-sample problem, the variance: 
admissibility of test for, 312; confidence 
intervals for, 217, 352; credible region for, 
229; likelihood ratio test for, 108; non- 
robustness of test for, 206; optimum test for, 
108, 139, 193, 290, 511 

Normal response model, 165 

Normal scores test, 318, 322, 323, 324, 357, 
360; comparison with t-test, 321; optimality 
of, 320 

Normal subgroup, 337 

Normal two-sample problem, difference of 
means: comparison with matched pairs, 
264; confidence intervals for, 218, 353; 
credible region for, 262; test for (variances 
equal), 122, 201, 204, 208, 255, 296, 373. 
See also Behrens-Fisher problem; 
Homogeneity, tests of; t-distribution; t-test; 
Two-sample problem 

Normal two-sample problem, ratio of 
variances: confidence intervals for, 21 8, 
333, 351; credible region for, 262; 
nonrobustness of test for, 207; test for, 122, 
198, 290. See also F-test for ratio of 
variances; Ratio of variances 

Null set, 48, 61, 140 


Odds ratio, 154, 163, 164, 547; most 
accurate unbiased confidence intervals for, 
261. See also Binomial probabilities; 
Contingency table; Two by two tables 

One-parameter exponential family, 80, 101; 
most stringent test in, 527. See also 
Exponential family 

One-sided hypotheses, 78, 151, 167; 
multivariate, 460. See also Confidence 
bounds 

One-way classification, 374; Bayesian 
inference for, 427; model II for, 418; 
multivariate, 463; nonparametric, 380. 
See also Homogeneity, tests of; Normal 
many-sample problem 
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Onto, see Transformation 

Optimality, ix, xii, 8, 9 

Orbit of transformation group, 285 

Ordered alternatives, 380 

Order statistics, 46; completeness of, 143, 
173, 183, 187; distribution of, 345; 
equivalent to sums of powers, 46; 
expected values of, 318; as maximal 
invariants, 286; in permutation tests, 231; 
as sufficient statistics, 63, 231 

Orthogonal group, 286, 366, 518 


Paired comparisons, see Matched pairs 

Pairwise sufficiency, 64 

Parameters, unrelated, see Variation 
independent parameters 

Parameter space, 1 

Pareto distribution, 123, 272 

Partial ancillarity, 546, 547, 561 

Partial sufficiency, 122, 565 

Performance robustness, 208, 321. See also 
Robustness 

Permutation test, 208, 232, 265, 273, 276, 
218, 279, 450; approximated by standard 
t-test, 236, 253; complete class, 243; 
confidence intervals based on, 246, 263, 
266, 267; most powerful for nonparametric 
hypotheses, 232, 252; as randomization 
test, 238; robustness of, 321; most 
stringent, 533; for testing independence, 
252; for variances, 378. See also 
Nonparametric; Randomization model 

Pillai-Bartlett trace test, 463; robustness of, 
465 

Pivotal quantity, 333, 357 

Point estimation, 4, 30; equivariant, 12; 
unbiased, 13, 14, 23. See also Median 
unbiasedness 

Poisson distribution P(r), 2, 56, 65, 171; as 
distribution of sum of Poisson variables, 
65; relation to exponential distribution, 
23, 82, 88, 114; square root transformation 
for, 432, 445; sufficient statistics for, 20. 
See also Exponential distribution; Poisson 
parameters; Poisson process 

Poisson model: for 2 X 2 table, 159, 161; 
for 2 X 2 X K table, 163, 181 

Poisson parameters: comparing k, 364; 
comparing two, 151, 152, 186, 221, 546; 
confidence intervals for the ratio of two, 
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Poisson parameters (Continued ) 
221; one-sided test for, 81, 114; 
one-sided test for sum of, 120 

Poisson process, 3, 65, 88; comparison of 
experiments for, 88; confidence bounds 
for scale parameter, 92; distribution of 
waiting times in, 23; test for scale 
parameter in, 81, 114; and 2 X 2 tables, 
159. See also Exponential distribution 

Polya frequency function, 509, 538. See 
also Total positivity 

Positive dependence, see Dependence, 
positive 

Positive part of a function, 38 

Posterior distribution, 225; percentiles of, 
229. See also Bayesian inference 

Posterior probability, 125 

Power function, 69; of invariant test, 300; of 
one-sided test, 79, 117; of two-sided test, 
102 

Power series distribution, 181 

Power of a test, 69, 70, 446; conditional, 
150, 547; robustness of, 207; unbiased 
estimation of, 151, 547 

Preference ordering of decision procedures, 9, 
14, 15 

Prior distribution, 14, 225; improper, 226, 
311; noninformative, 226. See also Bayesian 
inference; Least favorable distribution; 
Posterior distribution 

Probability density (with respect to x), 40; 
convergence theorem for, 573 

Probability distribution of a random variable, 
36. See also Cumulative distribution 
function (cdf) 

Probability integral transformation, 320 

Probability measure, 35 

Probability ratio, see Likelihood ratio 

Probability theory, 34, 66 

Product measure, 40 

Pavia : as maximal invariant, 287, 

Pseudometric space, 571 

P-value, 70, 114, 170; combination of, from 
independent experiments, 170. 


Quadrant dependence, 176, 251, 271. See 
also Dependence, positive 

Quadrinomial distribution, 163 

Quality control, 106, 293 
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Radon-Nikodym derivative, 40; properties of, 
61 

Radon-Nikodym theorem, 40 

Random assignment, 160, 161, 238, 396 

Random breaking of ties, 167 

Random effects model, 418, 426, 447; for 
nested classifications, 422; for one-way 
layout, 418; for two-way layout, 438, 440. 
See also Ratio of variances 

Randomization, 6, 396; as basis for inference, 
238; to lower the maximum risk, 25; 
possibility of dispensing with, 113; relation 
to permutation test, 240. See also Random 
assignment; Randomized procedure 

Randomization model, 162, 245; confidence 
intervals in, 246 

Randomized procedure, 6, 25, 1 13; confidence 
intervals, 219; test, 71, 74, 155 

Randomness, hypothesis of, 349, 350 

Random sample size, 112, 181, 561 

Random variable, 36 

Rank correlation coefficient, 351 

Ranks, 286; distribution under alternative, 
344, 345, 361; as maximal invariants, 286, 
315; null distribution of, 317. See also 
Signed ranks 

Rank-sum test, 178, 184. See also Wilcoxon 
test. 

Rank tests, 316; surveys of, 380. See also 
Independence, test for; Nonparametric; 
Nonparametric two-sample problem; 
Symmetry; Trend 

Ratio of quadratic forms, maximum of, 474 

Ratio of variances: confidence intervals for, 
219, 262, 333, 351; in model II, 419, 
421, 558; tests for, in two-sample 
problems, 122, 198, 207, 290, 339, 562. 
See also F-test for ratio of variances; 
Homogeneity, tests of; Random effects 
model 

Rectangular distribution, see Uniform 
distribution 

Reference set, ix. See also Conditional 
inference 

Regression, 222, 446, 450, 542; with both 
variables subject to error, 435; comparing 
several lines, 399, 435; confidence band for, 
417, 444; confidence intervals for 
coefficients, 223, 398; confidence sets for 
abscissa of line, 224; general linear model 
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for, 374, 430; as linear model, 365; 
multivariate, 462, 490, 496; nonparametric, 
350; polynomial, 435; robustness of tests 
for, 401, 436; tests for coefficients, 223, 
397, 398, 400. See also Trend 

Regression dependence, 251, 271, 315; See 
also Dependence, positive 

Relevant and semirelevant subsets, 230, 554, 
564, 568; randomized version of, 563 

Repeated measurements, 462, 466 

Restricted Bayes solution, 15, 30 

Restricted x2-test, 481, 500 

Risk function, 2, 28 

Robustness, ix, 10, 203, 208, 213, 273, 444, 
536; of analysis of variance tests, 401; 
against dependence, 209; for F-test of 
means, 378, 379; of general linear models 
tests, 379, 405; lack of, for F-test of 
variances, 207, 422; lack of, for x?-test of 
variance, 206; lack of, for Wilcoxon test, 
323; of multivariate tests, 465, 491; of 
regression tests, 401, 405; of test of 
independence or lack of correlation, 250; for 
tests in two-way layout, 434, 436; of t- 
test, 205, 209, 273, 321. See also 
Adaptive test; Behrens-Fisher problem; 
Efficiency robustness; Huber condition; 
Performance robustness; Permutation test; 
Rank tests 

Roy’s maximum root test, 463, 465; 
robustness of, 465; simultaneous 
confidence sets based on, 475 

Runs test: power of, 183; for testing 
independence in a Markov chain, 176, 
177 


Sample, 3; haphazard, 237; stratified, 
231 

Sample correlation coefficient R, 249; 
distribution of, 267, 270, 271, 276; 
monotone likelihood ratio of distribution, 
340; variance stabilizing transformation 
for, 432. See also Bivariate normal 
distribution; Multiple correlation 
coefficient; Rank correlation coefficient 

Sample distribution function, see Empirical 
distribution function (EDF) 

Sample inspection: by attributes, 80, 293, 
339; choice of inspection stringency for, 
89; for con:paring two products, 167, 296; 
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comparison of two methods, 339; by 
variables, 106, 293, 339 

Sample size: required to achieve specified 
power, 70, 153, 260, 504 

Sample space, 37 

S-ancillary, see Partial ancillary 

Scale families: condition for monotone 
likelihood ratio, 510 

Scheffé’s S-method, 382, 388, 405, 411, 
444; alternatives to, 417, 437; multivariate 
extensions, 471 

Selection procedures, 117, 127 

Separable: family of distributions, 574; space, 
571 

Separate families of hypotheses, 290, 338, 
355, 360, 363 

Sequential analysis, ix, 8, 78, 175, 196, 215 

Sequential experimentation, 8, 66 

Shift, confidence intervals for: based on 
permutation tests, 246, 263; based on rank 
tests, 347, 362. See also Behrens-Fisher 
problem; Exponential distribution; 
Nonparametric two-sample problem; 
Normal two-sample problem, difference of 
means 

Shift model, 164, 329 

c-field, 35; with countable generators, 575 

o-finite, 35 

Signed ranks, 317; distribution under 
alternatives, 348; null distribution of, 324 

Significance level, 69, 71; for multiple 
comparisons, 382, 385; nominal, 387. See 
also P-value 

Significance probability, See P-value 

Sign test, 106; in double exponential 
distribution, 531; for matched pairs, 170; 
for testing consumer preferences, 166; for 
testing symmetry with respect to a given 
point, 168, 325, 530; treatment of ties in, 
167, 186. See also Binomial probabilities; 
Median; Sample inspection 

Similar test, 135, 140, 182, 183, 186; 
characterization of, 144; relation to 
unbiased test, 135 

Simple: class of distributions, 72; hypothesis, 
73, 483 

Simple function, 37 

Simple hypothesis vs. simple alternative, 73; 
with large samples, 125. See also Neyman- 
Pearson fundamental lemma 
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Simultaneous confidence ellipsoids, 576 

Simultaneous confidence intervals, 388, 406, 
411, 444, 452; for the components of a 
vector mean, 411; for all contrasts, 388, 
415; in multivariate case, 471, 503. 
See also Confidence bands; Dunnett’s 
multiple comparison method; Scheffé’s 
S-method; Tukey’s T-method 

Simultaneous confidence sets: for a family of 
linear functions, 408; multivariate, 475, 498; 
smallest, 409; taut, 409 

Simultaneous inference, ix 

Simultaneous tests, 70, 415. See also Multiple 
comparisons 

Smirnov test, 322, 323 

Spherically symmetric distributions, 257, 439 

Square root transformation, 432, 445 

Stagewise tests, 381, 388 

Standard confidence bounds, 96, 229 

Stationarity, 176 

Statistic, 37; equivalent representations of, 41; 
fully informative, 113; subfield induced by, 
41 

Statistical inference, 1; and decision theory, 
4,71 

Stein’s two-stage procedure, 258 

Stochastically increasing, 84; relation to 
monotone likelihood ratio, 85 

Stochastically larger, 84, 116, 314 

Stochastic process, 129. See also Poisson 
process 

Stratified sampling, 231 

Strictly unbiased, 137 

Strongly unimodal, 509, 562 

Studentization, 209, 213, 380 

Studentized range, 381, 443 

Student's t-test, see t-test 

Subfield, 41 

Sufficient statistic, 19, 30, 53, 66, 67, 124; 
asymptotically, 485; Bayes definition of, 21, 
22; factorization criterion, 19, 30, 31, $3, 
54; likelihood ratio as, 63; minimal, 22, 28; 
pairwise, 64; in presence of nuisance 
parameters, 122; relation to ancillarity, 545; 
relation to comparison of experiments, 87; 
relation to fully informative statistic, 113; 
relation to invariance, 290, 301; statistics 
independent of, 191. See also Partial 
sufficiency 

Symmetric distribution, 63 


Symmetry, 10; relation to invariance, 11, 377; 
in a square two-way contingency table, 495; 
sufficient statistics for distributions with, 63; 
testing for, 326, 360, 361; testing, with 
respect to given point, 168, 316, 323, 325, 
326, 349 


Tautness, 409 

t-distribution, 196, 257, 258, 280; as 
approximation to permutation distribution, 
236; as distribution of function of sample 
correlation coefficient, 250; monotone 
likelihood ratio of, 295; multivariate, 353; 
noncentral, 196, 253, 276; normal limit of, 
205; as posterior distribution, 228; in two- 
stage-sampling, 259 

Test, 3, 68; almost invariant, 297; conditional, 
541, 549, 552; invariant, 284; locally 
maximin, 507; locally most powerful 
(LMP), 202, 527, 528, 538; maximin, 505; 
most stringent, 526; randomized, 71, 155; 
similar, 135; strictly unbiased, 137; of type 
A,131, 538; oftype A,, 131; oftype B, 202, 
538; of Type B,, 202; type D,E, 529; 
unbiased, 13, 134; uniformly most powerful 
(UMP), 32 

Three-decision problems, 101, 152 

Three factor contingency table, 162 

Ties, 167, 186 

Time series, 213 

Total positivity, 86, 118, 119, 140, 509; of 
order three, 119, 120, 303. See also Polya 
frequency function 

TPE, ix, x 

Transformation: of integrals, 43; into, 36; 
onto, 36; probability integral, 320; variance 
stabilizing, 376, 432, 433 

Transformation group, 570. See also 
Invariance 

Transitive: binary relation, 569; transformation 
group, 285 

Trend: test for absence of, 349, 403 

Triangular distribution, 355 

t-test: admissibility of, 309, 310, 343; as 
Bayes solution, 311, 343; comparison with 
Wilcoxon and Normal scores tests, 321, 
324; notefficiency robust, 322; as likelihood 
ratio test, 27, 108; in linear hypothesis with 
one constraint, 370; for matched pairs, 240, 
264; permutation version of, 208, 236; 
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power of, 196, 203, 207, 253, 256; one- 
sample, 111, 195, 209, 213, 257, 273, 339, 
380; for regression coefficient, 223, 397, 
398; relevant subsets for, 557; robustness 
of, 205, 207, 208, 209, 273; two-sample, 
202, 207, 230, 361; two-stage, 258. See 
also Normal one- and two-sample problem; 
Regression; Welch approximate t-test 

Tukey levels for multiple comparisons, 383, 
387, 433 

Tukey’s T-method, 382, 388, 433, 442, 443, 
451 

Two-sample problem, see Behrens-Fisher 
problem; Binomial probabilities; 
Exponential distribution; Matched pairs; 
Nonparametric two-sample problem; 
Normal two-sample problem; Permutation 
test; Poisson parameters; Shift, confidence 
intervals for; Two-by-two tables 

Two-sided alternatives, 101, 135, 152, 167 

Two-stage procedures, 258, 259 

Two by two tables: alternative models for, 
159, 161; comparison of experiments for, 
87, 159; Fisher’s exact test for, 155, 180, 
187; for matched pairs, 169, 179, 180; 
multinomial model for, 157; S-ancillaries 
for, 547, 568. See also Contingency tables 

Two by two by two table, 165 

Two-way classification: Bayesian inference 
for, 427; mixed model for, 439, 440, 441; 
with m observations per cell, 393; multiple 
comparison procedures for, 396; 
multivariate, 492, 493; with one observation 
per cell, 388; random effects model for, 
438, 440; rank tests for, 392; reorganization 
of variables in, 433; robustness of tests in, 
434, 436; simultaneous inference in, 416. 
See also Contingency tables; Interaction; 
Nested classification; Two-by-two tables 

Two-way contingency tables, see Contingency 
tables; Two-by-two tables 

Two-way layout, see Two-way classification 

Type A, A,, B, Bi, D, E test, see Test of type 
A, A, B, B}, D E 


UMP invariant test, 188, 289, 292; 
admissibility, 305; conditional, 551, 553; 
conditions to be UMP almost invariant, 
297; examples of nonuniqueness, 304, 305; 
relation with UMP unbiased test, 302. See 


also Invariance; Linear hypothesis, 
multivariate; Linear hypothesis, univariate 

UMP test, 72, 126; conditional, 542, 549, 
550, 552; examples involving two 
parameters, 112; for exponential 
distributions, 112; for inverse Gaussian 
distributions, 124; in monotone likelihood 
ratio families, 78; a nonparametric example, 
107; in normal one-sample problem, 108, 
111; in one-parameter exponential families, 
80; for uniform distributions, 111, 115; in 
Weibull distributions, 124 

UMP unbiased test, 134, 135, 186; 
admissibility of, 170; example of 
nonexistence of, 171; via invariance, 188, 
302; for multiparameter exponential 
families, 147, 188; for one-parameter 
exponential families, 135; for strictly totally 
positive families, 140. See also 
Unbiasedness 

Unbiasedness, 12, 23, 28, 186; for confidence 
sets, 13, 24, 216; and invariance, 24, 302; 
and minimax, 26; for point estimation, 13, 
23, 28; and similarity, 135; strict, 137; of 
tests, 134; for two-decision procedures, I3: 
See also UMP unbiased test; Uniformly 
most accurate confidence sets 

Undetermined multipliers, 100, 104, 118 

Uniform distribution U(a,b), 7, 21, 23; 
completeness of, 141, 172; discrete, 123, 
180; as distribution of integral transform, 
320; distribution of order statistics from, 
345; as null distributions of p-value, 170; 
one-sample problems in, 111, 115, 354, 
563; relation to exponential distribution, 
112; sufficient statistics for, 21, 28, 172; 
testing against exponential or triangular 
distribution, 355; other tests for, 480, 
482 

Uniformly most accurate confidence sets, 90, 
217; equivariant, 327, 524; relationto UMP 
tests, 91; unbiased, 217; uniformly 
minimize expected Lebesgue measure, 330. 
See also Confidence bands; Confidence 
bounds; Confidence intervals; Confidence 
sets; Simultaneous confidence intervals; 
Simultaneous confidence sets 

Uniformly most powerful, see UMP invariant 
test; UMP test; UMP unbiased test 

Unimodal, 562. See also Strongly unimodel 


600 


Unrelated parameters, see Variation 
independent parameters 


Variance components, see Components of 
variance 

Variance stabilizing transformation, 376, 432 

Variation diminishing, 86. See also Total 
positivity 

Variation independent parameters, 546, 561 


Waiting times (in a Poisson process), 23, 114. 
See also Exponential distribution; Life 
testing; Poisson process 

Weak compactness theorem, 576 

Weak convergence, 571, 572 

Weibull distribution W(b,c), 124, 567 
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Welch approximate t-test, 209, 304 

Welch-Aspin test, 304; relevant subsets for, 
558, 566 

Wilcoxon one-sample test, 324, 326, 348, 
349, 364 

Wilcoxon signed-rank test, see Wilcoxon one- 
sample test 

Wilcoxon two-sample test, 318, 322, 323, 
343, 357; comparison with t-test, 321; 
confidence intervals based on, 329; history 
of, 360, 364; optimality of, 320, 346 

Wilks’ A, 463; robustness of, 465 

Wishart distribution, 490 

Working-Hotelling confidence band, 417, 444 


Yule's measure of association, 157 
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